Jenkins Docker Integration: Build Pipelines That Don't Burn Down at 3 AM

Everyone thinks Dockerizing Jenkins is about 'consistency' and 'reproducibility.' That's the marketing fluff. The real reason is survival: static build slaves are ticking time bombs. I've watched a team's Jenkins master grind to a halt because one rogue build filled up the slave's disk with node_modules. With Docker, that build dies in isolation and the next one starts clean. This article covers the patterns that actually work in production — not the hello-world examples. You'll learn how to set up dynamic Docker agents, cache dependencies without leaking disk, and debug the inevitable 'container exited with code 137' at 3 AM.

Why Static Build Slaves Are a Legacy Anti-Pattern

Before Docker, Jenkins slaves were long-lived VMs or bare metal. You'd SSH in, install JDK 8, Maven, Node 12, Python 3.6 — and pray no build left behind a zombie process or filled the disk. The problem? State leaks. One build's npm install pollutes the next. A stuck mvn test eats all RAM. You end up with 'snowflake' slaves that can't be reproduced. Docker fixes this by giving each build a fresh container. But the real win is elasticity: you can spin up 20 containers on a single beefy host, or spread across a swarm/K8s cluster. No more 'waiting for executor' because a slave is busy with a different project.

// io.thecodeforge — DevOps tutorial

pipeline {
    agent none
    stages {
        stage('Build') {
            agent {
                docker {
                    image 'maven:3.8.6-eclipse-temurin-11'
                    args '-v /tmp/maven-cache:/root/.m2 --memory=2g'
                }
            }
            steps {
                sh 'mvn clean package'
            }
        }
    }
}

Dynamic Docker Agents: Spinning Up Ephemeral Workers

The simplest integration is the docker agent — Jenkins creates a container for the entire pipeline or per stage. But for heavy builds, you want per-stage containers to avoid image bloat. The Docker Pipeline plugin gives you docker.image('image').inside() for fine-grained control. Why would you use this? When your build needs Node 18, but your test stage needs Python 3.10. With static slaves, you'd need both installed. With Docker, each stage gets its own image. The trade-off: pulling images adds latency. Mitigate with a local registry mirror or pre-pull images on the host.

// io.thecodeforge — DevOps tutorial

pipeline {
    agent any
    stages {
        stage('Install Dependencies') {
            steps {
                script {
                    docker.image('node:18-alpine').inside('-v /tmp/npm-cache:/root/.npm') {
                        sh 'npm ci'
                    }
                }
            }
        }
        stage('Test') {
            steps {
                script {
                    docker.image('python:3.10-slim').inside {
                        sh 'pip install -r requirements.txt && pytest'
                    }
                }
            }
        }
    }
}

Docker-in-Docker: The Pattern That Works (and the One That Doesn't)

Sometimes your build needs to build Docker images itself — a CI pipeline that produces a Docker image. The naive approach is Docker-in-Docker (DinD): run a container with Docker inside. Don't. It's a security nightmare and requires privileged mode. Instead, mount the host's Docker socket into the container. This is called 'Docker-out-of-Docker' (DooD). The container talks to the host's Docker daemon. It's simpler, faster, and doesn't need privileged mode. The catch: containers spawned by the build appear on the host, not inside the container. That's fine — they're ephemeral anyway.

// io.thecodeforge — DevOps tutorial

pipeline {
    agent any
    stages {
        stage('Build Docker Image') {
            steps {
                script {
                    docker.image('docker:20.10.16-dind').inside(
                        '-v /var/run/docker.sock:/var/run/docker.sock'
                    ) {
                        sh 'docker build -t myapp:${BUILD_NUMBER} .'
                    }
                }
            }
        }
    }
}

Caching Dependencies Without Blowing Up Disk

The biggest pain with Docker in CI is image pull times and cache misses. You want to cache node_modules, .m2, pip packages, etc. The naive approach: mount a host volume. That works, but the cache grows unbounded. I've seen /tmp/npm-cache hit 50GB. Set a size limit with Docker's --storage-opt or use a cron job to prune old caches. Better: use a dedicated cache volume with docker volume create --driver local --opt type=none --opt device=/data/cache --opt o=bind cache_vol. Then mount it with -v cache_vol:/cache. This gives you control over the backing filesystem and can be backed up.

// io.thecodeforge — DevOps tutorial

pipeline {
    agent any
    environment {
        CACHE_VOL = 'maven-cache'
    }
    stages {
        stage('Build with Cache') {
            steps {
                script {
                    // Create cache volume if not exists
                    sh 'docker volume inspect ${CACHE_VOL} || docker volume create ${CACHE_VOL}'
                    docker.image('maven:3.8.6-eclipse-temurin-11').inside(
                        "-v ${CACHE_VOL}:/root/.m2 --memory=2g"
                    ) {
                        sh 'mvn clean package'
                    }
                }
            }
        }
    }
}

When Docker Agents Break: Debugging Container Failures

Containers fail silently. The most common error is exit code 137 (OOMKilled). You won't see it in the build logs — just 'Process exited with code 137'. Check dmesg | grep -i oom on the host. Another: exit code 139 (segfault) — usually a memory corruption or incompatible library. Exit code 1 is a script error. But the worst is exit code 0 with no output — the container started and immediately exited because the entrypoint ran and finished. This happens when you use docker run without a long-running process. Jenkins uses cat as the entrypoint to keep the container alive. If you override entrypoint, the container exits immediately.

// io.thecodeforge — DevOps tutorial

# Check OOM kills on the Docker host
sudo dmesg | grep -i oom | tail -20

# Check Docker container exit codes
# In Jenkins pipeline, add this step to debug:
sh 'docker inspect $(docker ps -aq --filter name=jenkins-) --format "{{.State.ExitCode}}"'

# Test if an image can run interactively
docker run --rm -it alpine:3.16 sh -c 'echo "Container works"; exit 0'

# If using custom entrypoint, ensure it doesn't exit
# Bad: CMD ["build.sh"]  # exits after build
# Good: CMD ["sh", "-c", "build.sh && tail -f /dev/null"]