🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles
dockercicdbeginners

What is Docker Layer Caching? Explained Simply

Docker layer caching can make your builds 10x faster or silently break them. Here's exactly how it works and how to use it properly.

DevOpsBoys4 min read
Share:Tweet

Docker layer caching is one of those things that either saves you 5 minutes per build or causes subtle bugs — depending on whether you understand it.

Every Docker Command Creates a Layer

When Docker builds an image, each instruction in the Dockerfile creates a layer — a snapshot of the filesystem at that point.

dockerfile
FROM ubuntu:22.04          # Layer 1
RUN apt-get update         # Layer 2
RUN apt-get install -y python3  # Layer 3
COPY app.py /app/          # Layer 4
RUN pip install -r requirements.txt  # Layer 5
CMD ["python3", "/app/app.py"]       # Layer 6

Docker stores each layer as a content-addressed hash. If a layer hasn't changed, Docker reuses the cached version instead of re-running the command.

The Cache Invalidation Rule

Here's the critical rule: if a layer changes, all subsequent layers are rebuilt.

Layer 1: FROM ubuntu:22.04  → cached ✓
Layer 2: RUN apt-get update → cached ✓
Layer 3: RUN apt-get install -y python3 → cached ✓
Layer 4: COPY app.py /app/ → CHANGED (you edited app.py)
Layer 5: pip install -r requirements.txt → REBUILT (even if requirements.txt didn't change)
Layer 6: CMD → REBUILT

This means layer order matters a lot for build speed.

The Classic Mistake (Slow Builds)

dockerfile
# BAD - copies everything first, then installs
FROM python:3.11
WORKDIR /app
COPY . .                           # copies all files including app.py
RUN pip install -r requirements.txt  # reinstalls every time any file changes

Every time you change app.py, Docker invalidates the COPY . . layer, which forces pip to reinstall all packages. A change to a single Python file causes a 3-minute pip install on every build.

The Correct Pattern (Fast Builds)

dockerfile
# GOOD - install dependencies first, copy code last
FROM python:3.11
WORKDIR /app
COPY requirements.txt .            # only this file
RUN pip install -r requirements.txt  # cached as long as requirements.txt doesn't change
COPY . .                           # copy app code last
CMD ["python", "app.py"]

Now pip install only re-runs when requirements.txt changes. Editing app.py just re-runs the COPY . . step — which is instant.

Same Pattern for Node.js

dockerfile
FROM node:20
WORKDIR /app
COPY package*.json ./      # copy package files first
RUN npm ci                 # install dependencies (cached)
COPY . .                   # copy app code
CMD ["node", "index.js"]

Same Pattern for Go

dockerfile
FROM golang:1.22
WORKDIR /app
COPY go.mod go.sum ./     # copy module files first
RUN go mod download       # download dependencies (cached)
COPY . .                  # copy source
RUN go build -o main .
CMD ["./main"]

How Caching Works in CI/CD

In local builds, Docker automatically caches layers between builds. In CI/CD, each build starts fresh — so you need to explicitly export and import the cache.

GitHub Actions with cache:

yaml
- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v3
 
- name: Build and push
  uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: myapp:latest
    cache-from: type=gha          # use GitHub Actions cache
    cache-to: type=gha,mode=max   # save to GitHub Actions cache

With registry cache (works across runners):

yaml
    cache-from: type=registry,ref=myrepo/myapp:buildcache
    cache-to: type=registry,ref=myrepo/myapp:buildcache,mode=max

--no-cache Flag

When you want to force a complete rebuild (e.g., to pick up security patches):

bash
docker build --no-cache -t myapp .

This ignores all cached layers and runs every instruction fresh.

Multi-Stage Builds and Caching

Multi-stage builds work great with caching. Each stage caches independently:

dockerfile
# Stage 1: dependencies (rarely changes)
FROM node:20 AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci
 
# Stage 2: build (changes when source changes)
FROM node:20 AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
 
# Stage 3: final image (just the output)
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html

The deps stage is cached as long as package*.json doesn't change — even when the source code changes.

Quick Checklist

  • Dependencies before source code (COPY requirements.txt before COPY . .)
  • Use COPY package*.json not COPY package.json (includes package-lock.json)
  • Set up cache in CI with type=gha or registry cache
  • Use --no-cache for security-sensitive builds
  • Use multi-stage builds to separate slow steps from fast ones

Docker layer caching is one of the easiest wins in build optimization. Fixing the COPY order alone can cut build times from 5 minutes to 30 seconds.

Learn more: Docker BuildKit caching, GitHub Actions Docker layer caching

Free Tools
Free Tool

Dockerfile Security Linter

Scan your Dockerfile for security issues, missing best practices, and optimization tips.

Free Tool

CI/CD Pipeline Generator

Generate GitHub Actions, GitLab CI, or Jenkins pipelines for your stack instantly.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Back to all articles
Share:Tweet
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments