Docker Compose for Production: The Hard Truths They Don't Tell You

You've been using Docker Compose for local dev. It's great. Now your boss says 'ship it to production with compose.' That's where the pain starts. I've seen a simple compose file bring down a payments service because someone forgot to set a memory limit — the container ate all the RAM, the kernel OOM-killed the database, and the entire platform went dark at 3 AM. Docker Compose for production is not docker-compose up -d. It's a completely different beast. By the end of this, you'll know exactly what to add, what to remove, and what to watch for when running compose in production. You'll be able to write a compose file that survives a traffic spike, a node failure, and a junior dev's 'quick fix'.

Why Your Dev Compose File Will Burn Production

The compose file you use for local development is optimized for convenience, not reliability. It probably has ports exposed to the world, no resource limits, and a restart policy of 'no'. In production, that's a disaster. I've seen a dev compose file with depends_on: - db and no health check — the app started before the database was ready, failed, and the restart policy wasn't set, so it stayed dead. The classic rookie mistake here costs you a full table lock on writes because the app connects before the DB is accepting queries. You need health checks, condition: service_healthy on depends_on, and proper restart policies. Don't assume order — enforce it.

// io.thecodeforge — DevOps tutorial

version: '3.8'

services:
  api:
    image: myapp/api:latest
    ports:
      - "127.0.0.1:8080:8080"  # Bind to localhost only — never 0.0.0.0 in production
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s  # Give app time to initialize
    depends_on:
      db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  db:
    image: postgres:15
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      resources:
        limits:
          memory: 1G

secrets:
  db_password:
    file: ./secrets/db_password.txt

volumes:
  pgdata:

Resource Limits: The One Thing Everyone Forgets

Without resource limits, a single container can consume all host memory and get OOM-killed, taking down other services. Worse, a CPU-hungry container can starve the rest. I've seen a log aggregator with a memory leak eat 8GB before the kernel killed it — and the database was on the same host. The fix: always set deploy.resources.limits.memory and deploy.resources.limits.cpus. Use reservations to guarantee minimum resources. For memory, also set --memory-swap to allow controlled swap usage. The rule of thumb: limit memory to 80% of what you expect the container to need under peak load. Monitor with docker stats and adjust.

// io.thecodeforge — DevOps tutorial

services:
  worker:
    image: myapp/worker:latest
    deploy:
      resources:
        limits:
          cpus: '1.0'        # One full core
          memory: 512M        # Hard limit
        reservations:
          cpus: '0.5'         # Guaranteed minimum
          memory: 256M
    # Optional: allow swap up to 1GB total (memory + swap)
    # memswap_limit: 1G
    # mem_swappiness: 0  # Disable swapping if possible

Health Checks: The Difference Between Self-Healing and Self-Destructing

A container is 'running' if its main process is alive. That doesn't mean your app is healthy. I've seen a Node.js app that was running but stuck in an infinite loop — the health endpoint never responded, but Docker thought it was fine. The load balancer kept sending traffic, users got timeouts, and the monitoring didn't catch it because the container was 'up'. Health checks fix this. They tell Docker (and orchestrators) when a container is truly ready. Use them in depends_on with condition: service_healthy to ensure startup order. Also use them in restart policies — if health check fails, Docker can restart the container. The key is setting a proper start_period: give your app time to initialize before health checks start.

// io.thecodeforge — DevOps tutorial

services:
  web:
    image: myapp/web:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s  # App may take 60s to start
    restart: unless-stopped
    # If health check fails 3 times, container is marked unhealthy
    # Docker will restart it (unless-stopped policy)
    # But only if the main process exits — health check alone doesn't trigger restart
    # For auto-restart on unhealthy, use a process manager or orchestration

Logging: Don't Let Logs Fill Your Disk

Default Docker logging uses json-file driver with no rotation. In production, a chatty app can fill a disk in hours. I've seen a misconfigured logger write 50GB of logs in a day, causing the host to run out of space, which killed all containers. The fix: configure logging driver with max-size and max-file. Better yet, use a centralized logging system (ELK, Loki, Datadog) with the syslog or fluentd driver. But even with centralized logging, always set local rotation as a safety net. Also consider log levels — in production, set to warn or error unless debugging.

// io.thecodeforge — DevOps tutorial

services:
  app:
    image: myapp/app:latest
    logging:
      driver: "json-file"
      options:
        max-size: "10m"   # Rotate when log file reaches 10MB
        max-file: "3"      # Keep 3 rotated files (30MB total)
        # Alternative: use syslog to send to central server
        # driver: "syslog"
        # options:
        #   syslog-address: "tcp://logs.example.com:514"
        #   tag: "{{.Name}}/{{.ID}}"

Secrets and Configs: Never Hardcode in Production

In dev, you might set environment variables directly in compose. In production, that's a security risk. Anyone with access to the compose file or the host can see secrets. Use Docker secrets (swarm mode) or bind-mount secret files. For compose without swarm, use the secrets: directive with file: or external: true. Never use environment: for passwords, API keys, or tokens. Also, use configs for non-sensitive configuration files. The pattern: write secrets to files outside the compose directory, set permissions to 600, and mount them. The app reads them at startup.

// io.thecodeforge — DevOps tutorial

services:
  api:
    image: myapp/api:latest
    secrets:
      - db_password
      - api_key
    configs:
      - source: app_config
        target: /etc/app/config.yml

secrets:
  db_password:
    file: ./secrets/db_password.txt  # Permissions 600
  api_key:
    external: true  # Created outside compose: docker secret create api_key -

configs:
  app_config:
    file: ./config/production.yml

Networking: Isolation and Service Discovery

In production, you don't want all containers on the same network. Isolate tiers: a frontend network, a backend network, and a database network. Only expose necessary ports. Use internal networks for databases. Docker Compose creates a default network, but you should define custom networks with driver: overlay for multi-host (swarm) or bridge for single host. Also, use container_name for stable DNS resolution. I've seen a service discovery fail because the container name changed after a restart — use container_name or service name (which is stable).

// io.thecodeforge — DevOps tutorial

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true  # No external access
  database:
    driver: bridge
    internal: true

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    networks:
      - frontend
      - backend

  api:
    image: myapp/api:latest
    networks:
      - backend
      - database
    # No frontend network — API not directly exposed

  db:
    image: postgres:15
    networks:
      - database
    # Only accessible from backend network

Volumes: Persistent Data and Performance

In production, database data must survive container restarts. Use named volumes, not bind mounts, for database data. Bind mounts are fine for configs but not for data — they have permission issues and are harder to back up. For performance, consider volume drivers like local (default) or use a network filesystem (NFS, EFS) for multi-host setups. Also, set volume labels for backup automation. I've seen a production database lose data because someone used a bind mount and the host directory got deleted. Named volumes are managed by Docker and safer.

// io.thecodeforge — DevOps tutorial

services:
  db:
    image: postgres:15
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro  # Bind mount for init script
    # Named volume for data

volumes:
  pgdata:
    # Use a driver for backups or replication
    # driver: local
    # driver_opts:
    #   type: none
    #   device: /path/to/backup
    #   o: bind

When Not to Use Docker Compose in Production

Docker Compose is great for single-host deployments or small clusters. But if you need multi-host orchestration, rolling updates, auto-scaling, or service discovery across nodes, you need Kubernetes, Docker Swarm, or Nomad. Compose doesn't handle node failures — if the host goes down, everything goes down. Also, Compose doesn't have built-in secrets management (without swarm), and its health check auto-recovery is limited. Use Compose when: you have a single server, you need simplicity, and you can tolerate downtime during updates. Use Kubernetes when: you need high availability, scaling, or multi-host deployments. The rule of thumb: if you have more than 3 services or need zero-downtime deploys, move to an orchestrator.

// io.thecodeforge — DevOps tutorial

# Signs you should NOT use Docker Compose in production:
# 1. You need rolling updates without downtime
# 2. You have multiple hosts and need service discovery
# 3. You need auto-scaling based on load
# 4. You need secrets management without swarm mode
# 5. You can't tolerate a single host failure
# 6. You need per-container resource metrics and alerts

# In these cases, use Kubernetes, Docker Swarm, or Nomad.

Rolling Updates with Compose? Not Really — But Here's the Hack

Docker Compose doesn't support rolling updates natively. docker-compose up -d recreates all containers at once, causing downtime. The hack: use docker-compose scale to run multiple replicas, then update one by one. But this is fragile. Better: use Docker Swarm with docker stack deploy, which supports rolling updates. Or use a reverse proxy (nginx, HAProxy) with blue-green deployment: start new containers on different ports, then switch traffic. I've used a simple script that updates containers in sequence with health checks between each. It's not perfect, but it works for low-traffic services.

// io.thecodeforge — DevOps tutorial

#!/bin/bash
# Simple rolling update for a service with 3 replicas

SERVICE="api"
REPLICAS=3

for i in $(seq 1 $REPLICAS); do
  echo "Updating replica $i..."
  docker-compose up -d --no-deps --scale $SERVICE=$((REPLICAS - i + 1)) --no-recreate $SERVICE
  # Wait for health check
  sleep 30
  # Check health
  if ! docker-compose ps | grep "$SERVICE.*Up"; then
    echo "Replica $i failed, rolling back..."
    docker-compose up -d --no-deps --scale $SERVICE=$REPLICAS
    exit 1
  fi
done

echo "All replicas updated successfully."