Search... ⌘K
Write Log In Sign Up

Join DevSolved

Share war stories & investigate outages.

Sign Up for Free Log In to Account
ESC
NORMAL Resolved

Docker Build Bloat: The 8GB Container Pipeline Bottleneck

Avatar Devsolved Jan 5, 2024 — 11:00 UTC MTTR: 4 hours 1 min read 75000 views
Executive Summary

Our CI/CD pipeline slowed down to a crawl, taking 45 minutes to deploy a minor CSS change because our Node.js Docker images had ballooned to 8GB each.

The Symptom

Symptom

Developers complained that GitLab CI pipelines were taking over 45 minutes just to push to the registry. EC2 nodes in the staging cluster were constantly running out of disk space (`No space left on device`).

Root Cause Analysis

Root Cause

The Dockerfile was poorly structured. It copied the entire directory (including `node_modules` and `.git`), ran `npm install`, and then ran `npm run build`. Every single file change invalidated the Docker layer cache, forcing a complete re-download of all dependencies. Furthermore, the base image was the bloated `node:18` (which is almost 1GB) instead of Alpine, and the `.dockerignore` file was completely missing.

Code dockerfile
FROM node:18
# The fatal flaw: Copying everything before installing invalidates cache!
COPY . .
RUN npm install
RUN npm run build
CMD ["npm", "start"]

Resolution

Resolution

We implemented a Multi-stage build using `node:18-alpine`. We explicitly copied `package.json` first to leverage layer caching, installed dependencies, built the app, and then copied ONLY the `dist/` folder into a pristine, tiny production image.
Code dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:18-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/main.js"]

Preventive Action Items

high Implement multi-stage builds across all 15 microservices @DevOps
normal Add mandatory `.dockerignore` reviews to PR templates @DevOps
|

Discussions 0

Most recent