What Actually Happens During a Container Build

Containers seem like magic: write a Dockerfile, run a build, and get an isolated application. But understanding the internals reveals elegant Linux primitives at work.

The Dockerfile Illusion

You write a Dockerfile. You run . A few minutes later, you have a container image that runs identically on your laptop, in CI, and in production. The abstraction is so effective that many developers never look beneath it. But underneath the Dockerfile syntax and the build output lies a fascinating set of mechanisms rooted in Linux kernel features, filesystem tricks, and a carefully designed layering system.

Understanding what actually happens during a container build makes you better at writing efficient Dockerfiles, debugging build failures, and making informed decisions about container security and performance.

Images Are Layers

A container image is not a single file. It is an ordered stack of filesystem layers, each representing a set of changes from the layer below it. When you build an image from a Dockerfile, each instruction (FROM, RUN, COPY, etc.) typically creates a new layer.

Consider a simple Dockerfile:

This produces roughly three meaningful layers: the base Ubuntu layer (pulled from a registry), the layer containing the installed Python packages, and the layer containing your application code. The CMD instruction sets metadata but does not create a filesystem layer.

Each layer is stored as a tar archive of the filesystem changes it introduces. A layer might add files, modify files, or delete files (deletions are recorded as "whiteout" markers). When the container runs, these layers are stacked using a union filesystem (commonly OverlayFS on modern Linux), presenting a unified view of the filesystem to the container process.

The Build Process Step by Step

When you run , here is what happens for each instruction in the Dockerfile:

1. Create a temporary container. The build engine starts a container from the image produced by the previous instruction (or the base image for the first instruction).

2. Execute the instruction. For a RUN instruction, the specified command is executed inside the temporary container. For a COPY instruction, files from the build context are copied into the container's filesystem. For ENV, LABEL, and similar instructions, metadata is recorded.

3. Snapshot the filesystem. After the instruction completes, the build engine takes a snapshot of the container's filesystem, capturing everything that changed. This diff becomes the new layer.

4. Commit the layer. The layer is saved, and a new intermediate image is created combining all previous layers plus the new one. The temporary container is removed.

5. Repeat. The process continues for each instruction until the Dockerfile is complete.

The final image is the combination of all layers plus the metadata from instructions like CMD, ENV, EXPOSE, and ENTRYPOINT.

Build Context

Before the build even starts, Docker collects the "build context": the set of files and directories available to COPY and ADD instructions. By default, this is the directory containing the Dockerfile. The entire build context is sent to the Docker daemon (or build engine) at the start of the build.

This is why a file matters. If your project directory contains large files, build artifacts, or a directory, all of that gets sent to the daemon, even if your Dockerfile never references it. A well-crafted can dramatically reduce build times and prevent sensitive files from accidentally ending up in your image.

Layer Caching

One of the most important optimizations in container builds is layer caching. The build engine checks whether it can reuse a cached layer for each instruction. The logic works roughly like this:

For a RUN instruction, the cache key is the command string. If the same command was run on the same parent layer in a previous build, the cached layer is reused.

For a COPY instruction, the cache key includes the checksums of the files being copied. If the files have not changed, the cached layer is reused.

Cache invalidation cascades: if one layer's cache is invalidated, all subsequent layers must be rebuilt, even if their instructions have not changed. This is why Dockerfile ordering matters. Instructions that change frequently (like copying application code) should come after instructions that change rarely (like installing system packages). If you put COPY before RUN apt-get install, any code change invalidates the package installation cache and forces a full reinstall.

Multi-Stage Builds

A powerful technique for producing smaller, more secure images is the multi-stage build. A Dockerfile can contain multiple FROM instructions, each starting a new build stage. You can copy files from one stage to another, leaving behind everything else.

The first stage installs dependencies and builds the application. The second stage starts fresh from a minimal Nginx image and copies only the built artifacts. The final image contains Nginx and the static files but none of the Node.js runtime, npm packages, or source code. This can reduce image sizes from gigabytes to megabytes.

Multi-stage builds also improve security by reducing the attack surface. Development tools, compilers, and build dependencies do not belong in production images, and multi-stage builds make it straightforward to exclude them.

Union Filesystems

The layered image model depends on union filesystems, which present multiple directories as a single merged view. On modern Linux, OverlayFS is the most common choice.

OverlayFS works with four directories: a lower directory (read-only layers), an upper directory (the writable layer), a work directory (used internally), and a merged directory (the unified view). When a process reads a file, OverlayFS checks the upper directory first, then falls through to the lower directories. When a process writes a file, the change goes to the upper directory. When a process deletes a file, a whiteout marker is created in the upper directory, hiding the file in the lower layers without actually modifying them.

This copy-on-write mechanism is what makes containers efficient. Running ten containers from the same image shares the read-only layers. Only the writes unique to each container consume additional storage.

Namespaces and cgroups

While container images are about packaging, container runtime isolation is about Linux kernel features: namespaces and cgroups.

Namespaces provide isolation. A PID namespace gives the container its own process tree, where the container's init process is PID 1. A network namespace gives it its own network interfaces and routing tables. A mount namespace gives it its own filesystem mounts. A user namespace can map container root to an unprivileged host user. There are also namespaces for IPC, UTS (hostname), and (in newer kernels) time.

Cgroups (control groups) provide resource limits. They allow you to restrict how much CPU, memory, I/O bandwidth, and other resources a container can use. Without cgroups, a runaway container could consume all host resources.

Together, namespaces and cgroups create the illusion of an isolated machine. But it is important to understand that this is not virtualization. All containers share the host's kernel. A kernel vulnerability can potentially allow a container to escape its isolation. This is why container security is a layered discipline that includes image scanning, runtime policies, and kernel hardening in addition to the basic isolation provided by namespaces and cgroups.

BuildKit and the Modern Build Engine

Docker's original build engine processed Dockerfiles sequentially, one instruction at a time. BuildKit, which became the default builder in Docker 23.0, introduced several improvements.

BuildKit can parallelize independent build stages. If two stages do not depend on each other, they build simultaneously. It supports more sophisticated caching, including the ability to mount caches that persist across builds (useful for package manager caches). It produces better progress output, showing concurrent operations clearly.

BuildKit also supports alternative frontends. While the Dockerfile syntax remains dominant, BuildKit can accept other formats through its LLB (low-level build) intermediate representation. This opens the door to build tools that produce optimized build graphs from higher-level descriptions.

Practical Implications

Understanding the build process leads to practical improvements in how you write Dockerfiles.

Order instructions from least to most frequently changing. Base image and system package installation should come first, application dependencies next, and application code last. This maximizes cache reuse.

Combine related RUN commands. Each RUN creates a layer, and layers include everything that changed during the command, including intermediate files. A pattern like installs packages and cleans up in a single layer, keeping the image small.

Use multi-stage builds to separate build-time and runtime dependencies. Your production image should contain only what is needed to run the application.

Use aggressively. Exclude everything that the build does not need.

Be aware of what goes into each layer. Running on an image shows the layers and their sizes, which can help identify unexpectedly large layers.

Build Caching and Layer Optimization

Understanding Docker's build cache is essential for writing efficient Dockerfiles. Docker caches each layer and reuses it if the instruction and all previous layers are unchanged. This means that Dockerfile order directly affects build speed.

A common pattern for Node.js applications illustrates the principle:

COPY package.json package-lock.json ./ (changes rarely) RUN npm install (cached when package files unchanged) COPY . . (changes frequently) RUN npm run build

By copying dependency files and installing dependencies before copying application code, the expensive npm install step is cached for most builds. If you reversed the order (COPY . . first), every code change would invalidate the npm install cache.

Multi-stage builds, introduced in Docker 17.05, allow a single Dockerfile to use multiple FROM statements. A typical pattern uses one stage for building (with compilers, build tools, and development dependencies) and a final stage for the runtime image (with only the compiled artifacts and production dependencies). This produces smaller, more secure images by excluding build tools from the final image.

BuildKit, Docker's next-generation build engine (default since Docker 23.0), adds parallel layer building, improved caching, and secret mounting (for accessing credentials during build without embedding them in layers). BuildKit's cache import/export feature allows CI/CD pipelines to share layer caches between builds, dramatically reducing build times for large images.

Security Considerations in Container Builds

Container images inherit the security properties of every layer in their build chain. A single vulnerable package in any layer creates a vulnerability in the final image. Security scanning at build time (using tools like Trivy, Grype, or Docker Scout) identifies known vulnerabilities before images are deployed.

Common security mistakes in Dockerfiles include running as root (the default), embedding secrets (API keys, passwords) in layers, using unversioned base images (FROM python:latest instead of FROM python:3.11.9), and installing unnecessary packages that expand the attack surface.

Distroless images (maintained by Google) and scratch-based images eliminate the operating system entirely, including only the application binary and its runtime dependencies. These minimal images have dramatically smaller attack surfaces: if there is no shell, an attacker who compromises the application cannot easily escalate to system-level access.

Container Registries and Distribution

Once built, container images must be distributed to the systems that run them. Container registries (Docker Hub, Amazon ECR, Google Artifact Registry, GitHub Container Registry) store and distribute images using the OCI Distribution Specification.

When you push an image to a registry, Docker uploads each layer individually (identified by its content hash) and a manifest that describes the image s configuration and layer ordering. When pulling, the runtime downloads only the layers it does not already have locally. This content-addressable storage makes distribution efficient: layers shared across images (such as a common base OS layer) are stored and transferred only once.

Image tags (like myapp:v1.2.3 or myapp:latest) are mutable pointers to image manifests. This mutability is both a feature and a hazard: pulling myapp:latest today and tomorrow may produce different images if the tag has been updated. For reproducibility, production systems reference images by their immutable digest (myapp@sha256:abc123...), which guarantees exactly the same image regardless of when or where it is pulled.

Image signing (using Cosign, Notary, or Docker Content Trust) provides cryptographic verification that an image was built by a trusted party and has not been tampered with since signing. Supply chain security frameworks like SLSA (Supply-chain Levels for Software Artifacts) define levels of assurance for the build process, from basic provenance tracking to fully hermetic, reproducible builds.

Runtime: From Image to Running Container

The container build produces an image. The container runtime transforms that image into a running process. Understanding this transformation clarifies the relationship between build-time and run-time concerns.

When you run a container, the runtime unpacks the image layers into a union filesystem, creates Linux namespaces (PID, network, mount, user, UTS, IPC) to isolate the process, applies cgroup limits (CPU, memory, I/O), sets up networking (virtual ethernet pairs, bridge networks, port mappings), and executes the container s entrypoint command.

The union filesystem (OverlayFS on most Linux distributions) layers the read-only image layers with a read-write layer on top. Writes go to the top layer only. Reads check the top layer first, then descend through the image layers. Deletes create whiteout files in the top layer. This architecture means the image layers are never modified by a running container, enabling multiple containers to share the same base layers while maintaining isolation between their writes.

Health checks, defined in the Dockerfile with the HEALTHCHECK instruction, allow the runtime to monitor the container s application-level health. Unlike process monitoring (which only detects crashes), health checks verify that the application is responding correctly. Container orchestrators like Kubernetes use health checks (liveness probes and readiness probes) to automatically restart unhealthy containers and route traffic away from containers that are not ready to serve requests.

OCI and the Future of Container Standards

The Open Container Initiative (OCI), founded in 2015 under the Linux Foundation, defines three specifications: the Image Specification (how images are structured), the Runtime Specification (how containers are executed), and the Distribution Specification (how images are stored and transferred). These standards ensure interoperability across container tools: an image built with Docker can be run by containerd, Podman, or CRI-O without modification.

Buildah, an OCI-compliant image builder, allows building images without a Docker daemon, which is valuable in CI/CD environments where running a privileged Docker daemon poses security risks. Kaniko (from Google) builds images inside Kubernetes pods without privileged access. These tools demonstrate how open standards enable a diverse ecosystem of interoperable tools, each optimized for different use cases.

Key Takeaways

Beyond Docker

The container ecosystem has evolved beyond Docker. Tools like Buildah, Podman, and Kaniko can build container images without a Docker daemon. The Open Container Initiative (OCI) has standardized image and runtime formats, ensuring interoperability. But the fundamental concepts, layers, union filesystems, namespaces, and cgroups, remain the same regardless of which tool you use.

Understanding these fundamentals gives you a foundation that transcends any particular tool or platform. Containers are not magic. They are a clever combination of Linux primitives, filesystem tricks, and a well-designed packaging format. And once you understand how they work, you can use them far more effectively.