Layer Cache and Best Practices

Dockerfile layer cache Understanding the layer cache is what tells you why Dockerfile instruction order affects build speed.

Core principles

Each instruction is a layer, the unit of caching. Docker compares the instruction string and relevant file checksums; only an exact match hits the cache.
Invalidation cascades: when one layer’s cache is invalidated, every layer after it rebuilds, whether or not they changed.

Invalidation conditions per instruction:

Instruction	Triggers invalidation
`FROM`	The base image digest changes
`RUN`	The instruction string changes (not the actual output: `apt-get update` pulling new versions does not invalidate the cache)
`COPY` / `ADD`	The source file checksum changes (mtime does not count)
`ENV` / `ARG` / `WORKDIR`, etc.	The instruction string changes

Instruction ordering

Principle: low-churn instructions first, high-churn last.

# Inefficient (source change → reinstall deps)
COPY . .
RUN pip install -r requirements.txt

# Optimized (only a dependency change invalidates)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

The RUN apt-get update trap: keep update and install in the same RUN, otherwise the cached update layer can leave install pulling stale versions:

RUN apt-get update && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

Forcing a skip and cleaning up

docker build --no-cache -t myapp .              # skip cache for the whole build
docker build --no-cache-filter install -t myapp .   # skip only one stage
docker builder prune                            # clear build cache

Cache mount (persist package cache across builds)

Persist package download cache across builds (not in an image layer):

# pip
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# npm
RUN --mount=type=cache,target=/root/.npm \
    npm install

# apt
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    apt-get update && apt-get install -y gcc

--mount=type=cache keeps the cache out of image layers and only persists it on the builder, saving build time over the traditional approach.

Best-practice template

Collected into a template you can copy directly:

# syntax=docker/dockerfile:1
FROM python:3.12-slim

LABEL org.opencontainers.image.authors="dev@example.com"

# Merge RUN, clean apt cache afterward: one fewer layer and less space
RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Create a non-root user
RUN groupadd -r appuser && useradd --no-log-init -r -g appuser appuser

WORKDIR /app

# Manifest first, source last (layer cache optimization)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY --chown=appuser:appuser . .

# Run as non-root
USER appuser

EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1

CMD ["python", "app.py"]

Key points: merge RUN and clean apt cache (rm -rf /var/lib/apt/lists/*), pip install --no-cache-dir to keep pip cache out of the image, pin the base tag, switch to a non-root USER, use .dockerignore to shrink the context, and use WORKDIR with absolute paths instead of RUN cd.

Notes and security

Avoid latest: pin the base to major.minor (or stricter, a digest @sha256:...) so the base does not silently change versions.
Switch to a non-root USER: running containers as root by default is risky; create a dedicated user and switch to it.
Keep secrets out of the Dockerfile: ARG / ENV stay in docker history; inject at runtime (docker run --env / Compose env_file / secrets), not hardcoded.
Be careful with ADD from URLs: caching and auth are complex; for downloads prefer RUN curl with a checksum, and for plain copies prefer COPY.

Dockerfile instructions one by one: USER, HEALTHCHECK, and other details.
A Python app Dockerfile: multi-stage to shrink the image further.

Reference: docs.docker.com/build/cache, build best practices

Overview

AI Models & Algorithms

Docker

Software & Tools

Knowledge & Method

Networking & Self-hosting

Layer Cache and Best Practices

Core principles

Instruction ordering

Forcing a skip and cleaning up

Cache mount (persist package cache across builds)

Best-practice template

Notes and security

Next

​Core principles

​Instruction ordering

​Forcing a skip and cleaning up

​Cache mount (persist package cache across builds)

​Best-practice template

​Notes and security

​Next

Core principles

Instruction ordering

Forcing a skip and cleaning up

Cache mount (persist package cache across builds)

Best-practice template

Notes and security

Next