Docker Storage and Volumes: The Real Deal on Data Persistence in Containers

Here's a truth that'll save your weekend: containers are designed to be disposable. That's the whole point — you kill them, you replace them, you scale them. But data doesn't scale. And when your database container restarts and all your customer orders vanish, you'll learn the hard way that Docker's default storage is a lie.

The problem is simple: every container starts with a fresh filesystem. Write to it, and that data lives only as long as the container. The moment it dies — crash, update, reschedule — your data dies too. That's fine for stateless apps. For anything else, you need storage that outlives the container.

By the end of this, you'll know exactly when to use volumes, bind mounts, and tmpfs. You'll understand the performance trade-offs, the backup strategies, and the gotchas that have burned teams in production. No fluff. Just the stuff that matters when your container goes down at 3 AM.

The Three Storage Options: When Each One Saves Your Bacon

Docker gives you three ways to persist data: volumes, bind mounts, and tmpfs. Each solves a different problem. Volumes are Docker-managed — they live in /var/lib/docker/volumes/ and are the safest default. Bind mounts map any host directory into the container — great for development, dangerous in production because they depend on host filesystem structure. tmpfs stores data in memory — fast but volatile, perfect for secrets or cache that must never hit disk.

Here's the rule: if you're storing data that must survive container restarts, use a volume. If you need to share config files from the host, use a bind mount. If you need a scratch space that's fast and disposable, use tmpfs. Anything else is asking for trouble.

// io.thecodeforge — DevOps tutorial

# Create a named volume
docker volume create myapp_data

# Run container with volume
docker run -d --name app1 -v myapp_data:/data alpine sleep 3600

# Run container with bind mount
docker run -d --name app2 --mount type=bind,source=/host/data,target=/data alpine sleep 3600

# Run container with tmpfs
docker run -d --name app3 --tmpfs /data:size=100M alpine sleep 3600

# Verify mounts
docker inspect app1 --format '{{json .Mounts}}' | jq .
docker inspect app2 --format '{{json .Mounts}}' | jq .
docker inspect app3 --format '{{json .Mounts}}' | jq .

How Docker's Layered Filesystem Actually Works (And Why It Lies to You)

Every Docker image is a stack of read-only layers. When you run a container, Docker adds a thin writable layer on top. This is the container's writable layer — it's where all your runtime changes go. But here's the catch: that layer is ephemeral. Delete the container, and the layer is gone. Worse, it's slow for write-heavy workloads because of copy-on-write semantics.

When you write to a file in the container, Docker copies it up from the image layer into the writable layer first. That's the copy-on-write penalty. For databases or any write-intensive app, this kills performance. That's why you mount a volume — it bypasses the layered filesystem entirely. The volume is a direct mount, no COW overhead.

This is also why docker commit is a terrible idea for backups. It captures the writable layer, but it's a snapshot, not a backup strategy. Use volumes and backup those instead.

// io.thecodeforge — DevOps tutorial

# Pull an image and inspect its layers
docker pull alpine:latest
docker history alpine:latest

# Run a container and see the writable layer
docker run -d --name layer_demo alpine sleep 3600
docker diff layer_demo  # Shows changes in writable layer

# Add a file and see the diff
docker exec layer_demo touch /tmp/test.txt
docker diff layer_demo  # Now shows A /tmp/test.txt

# Remove container — writable layer is gone
docker rm -f layer_demo

Volumes: The Production Workhorse — How to Create, Manage, and Backup

Volumes are Docker's first-class citizen for data persistence. They're managed by Docker, stored in /var/lib/docker/volumes/, and can be shared across containers. They work on any host, regardless of filesystem structure. They support volume drivers for NFS, cloud storage, and encryption.

Creating a volume is simple: docker volume create myvolume. But in production, you'll want to use the --mount syntax for clarity: docker run --mount type=volume,source=myvolume,target=/data. This makes it explicit that you're using a volume, not a bind mount.

Backup strategy: volumes are just directories on the host. You can tar them up, rsync them, or use a backup tool. For databases, always use the database's native backup tool (pg_dump, mysqldump) rather than copying volume files — you'll get consistent backups without locking issues.

// io.thecodeforge — DevOps tutorial

# Create a volume
docker volume create app_data

# List volumes
docker volume ls

# Inspect volume (shows mountpoint)
docker volume inspect app_data

# Run container with volume
docker run -d --name app -v app_data:/data alpine sleep 3600

# Backup volume to a tar file
docker run --rm -v app_data:/source -v $(pwd):/backup alpine tar czf /backup/app_data_backup.tar.gz -C /source .

# Restore volume from tar
docker run --rm -v app_data:/target -v $(pwd):/backup alpine sh -c "cd /target && tar xzf /backup/app_data_backup.tar.gz"

# Remove unused volumes
docker volume prune

Bind Mounts: The Double-Edged Sword for Development

Bind mounts map a host directory directly into the container. They're perfect for development — you can edit code on your host and see changes instantly in the container. But they're a security risk in production: the container can modify any file on the host, and the host filesystem structure becomes a dependency.

The classic rookie mistake: using a bind mount for a database data directory in production. When the host path changes or the directory is accidentally deleted, your database is gone. Volumes don't have this problem — they're managed by Docker and survive host directory changes.

For development, bind mounts are fine. Just use the :ro flag to mount read-only if you don't need write access: -v /host/path:/container/path:ro. This prevents accidental modifications from the container.

// io.thecodeforge — DevOps tutorial

# Development: mount source code for hot-reload
docker run -d -p 3000:3000 -v $(pwd)/src:/app/src:ro node:18 npm run dev

# Production trap: don't do this
docker run -d -v /host/data:/var/lib/mysql mysql  # BAD: host path dependency

# Safer: use named volume
docker run -d -v mysql_data:/var/lib/mysql mysql  # GOOD

# Check mount type
docker inspect container_name --format '{{range .Mounts}}{{.Type}}{{end}}'

tmpfs: The In-Memory Scratch Space You're Not Using Enough

tmpfs mounts store data in the container's memory. They're fast — RAM speed — and they disappear when the container stops. Perfect for secrets, temporary files, or cache that must never persist to disk. But they count against the container's memory limit, so don't use them for large datasets.

Use tmpfs when you need a scratch directory that's automatically cleaned up. For example, a CI build container that downloads dependencies, compiles, and then exits — the tmpfs ensures no leftover files. Or a web server that writes session files — tmpfs keeps them fast and ephemeral.

One gotcha: tmpfs is not shared between containers. Each container gets its own tmpfs mount. If you need shared memory, use a volume or bind mount.

// io.thecodeforge — DevOps tutorial

# Run container with tmpfs mount (100MB limit)
docker run --rm --tmpfs /scratch:size=100M alpine sh -c "dd if=/dev/zero of=/scratch/test bs=1M count=50 && ls -lh /scratch/test"

# Verify tmpfs mount
docker run --rm --tmpfs /tmp:size=50M alpine mount | grep tmpfs

# Use tmpfs for secrets (safer than env vars)
docker run --rm --tmpfs /secrets:size=1M -e DB_PASS_FILE=/secrets/db_pass alpine sh -c "echo 's3cr3t' > /secrets/db_pass && cat /secrets/db_pass"

Volume Drivers: When Local Storage Isn't Enough

Docker's volume plugin architecture lets you use remote storage backends: NFS, cloud block storage (EBS, Azure Disk), or distributed filesystems (GlusterFS, Ceph). This is essential for multi-host setups like Docker Swarm or Kubernetes (though K8s has its own volume system).

To use an NFS volume, install the docker-volume-netshare plugin or use the built-in local driver with NFS options. For cloud volumes, use the respective plugin (e.g., rexray for EBS).

The trade-off: network storage adds latency. For databases, local volumes with replication are usually better than network volumes. But for shared config or logs, network volumes simplify management.

// io.thecodeforge — DevOps tutorial

# Create an NFS volume (requires nfs-common on host)
docker volume create --driver local --opt type=nfs --opt o=addr=192.168.1.100,rw --opt device=:/exported/path nfs_volume

# Run container with NFS volume
docker run -d --mount source=nfs_volume,target=/data alpine sleep 3600

# Check volume details
docker volume inspect nfs_volume

# For cloud EBS (using rexray plugin)
# docker volume create --driver rexray --opt size=10 --opt volumetype=gp2 my_ebs_volume

Storage Drivers: Overlay2, Devicemapper, and When to Care

Docker uses a storage driver to manage the layered filesystem. The default on modern Linux is overlay2 — it's fast, stable, and uses copy-on-write efficiently. On older systems, you might see devicemapper or aufs. If you're still on devicemapper, upgrade — it's slow and has known issues with space reclamation.

You can check your storage driver with docker info | grep 'Storage Driver'. If it's not overlay2, consider migrating. Overlay2 supports page cache sharing between containers — if two containers use the same base image, they share the same cached pages, saving memory.

One gotcha: overlay2 has a limit on the number of lower layers (usually 128). If you have a deep image history, you might hit this limit. Keep your Dockerfiles lean — combine RUN commands and use multi-stage builds.

// io.thecodeforge — DevOps tutorial

# Check current storage driver
docker info | grep -i 'storage driver'

# Check disk usage by storage driver
docker system df

# For overlay2, check backing filesystem
df -h /var/lib/docker

# If using devicemapper, check loop device usage
docker info | grep -i 'loop'

Backup and Restore Strategies That Won't Fail You at 3 AM

Backing up Docker volumes is straightforward, but there are traps. Never backup a volume while the container is writing to it — you'll get inconsistent data. For databases, use the database's native backup tool. For application data, stop the container or use a filesystem snapshot.

The simplest backup: docker run --rm -v volume_name:/data -v $(pwd):/backup alpine tar czf /backup/backup.tar.gz -C /data .. This spins up a temporary container that mounts the volume and creates a tar. For restore, reverse the process.

For automated backups, use a cron job or a backup tool like Duplicati or restic. Restic supports Docker volumes directly with restic backup /var/lib/docker/volumes/volume_name/_data. Just make sure to stop the container first or use a snapshot.

// io.thecodeforge — DevOps tutorial

# Backup a volume (stop container first)
docker stop myapp
docker run --rm -v myapp_data:/data -v $(pwd):/backup alpine tar czf /backup/myapp_backup_$(date +%Y%m%d).tar.gz -C /data .
docker start myapp

# Restore a volume (stop container first)
docker stop myapp
docker run --rm -v myapp_data:/data -v $(pwd):/backup alpine sh -c "cd /data && tar xzf /backup/myapp_backup_20231001.tar.gz"
docker start myapp

# Using restic (example)
# restic -r /backup/restic-repo backup /var/lib/docker/volumes/myapp_data/_data

Performance Benchmarks: Volumes vs Bind Mounts vs tmpfs

Performance varies significantly between storage options. tmpfs is fastest (RAM speed), followed by volumes (direct mount, no COW), then bind mounts (similar to volumes but with host filesystem overhead), and finally the container's writable layer (slowest due to COW).

For write-heavy workloads like databases, always use a volume. The writable layer's COW penalty can be 10-100x slower for random writes. Bind mounts are comparable to volumes for reads, but writes go through the host's VFS layer, which can add latency on some filesystems.

tmpfs is ideal for ephemeral high-speed I/O, but it consumes RAM. If your container has a 512MB memory limit, a 256MB tmpfs leaves only 256MB for the application. Plan accordingly.

// io.thecodeforge — DevOps tutorial

# Test write performance to writable layer
docker run --rm alpine sh -c "dd if=/dev/zero of=/tmp/test bs=1M count=100"

# Test write performance to volume
docker run --rm -v test_vol:/data alpine sh -c "dd if=/dev/zero of=/data/test bs=1M count=100"

# Test write performance to tmpfs
docker run --rm --tmpfs /data:size=200M alpine sh -c "dd if=/dev/zero of=/data/test bs=1M count=100"

# Compare speeds (output shows MB/s)

When Not to Use Docker Volumes: The Overkill Scenarios

Volumes are great, but they're not always the answer. If you're running a stateless app that doesn't persist data, don't add a volume — you're adding complexity for no benefit. If you need to share data between containers on the same host, a volume works, but consider if a shared tmpfs or a simple bind mount is simpler.

For one-off scripts or CI jobs that process data and exit, tmpfs is often better — no cleanup needed. For development, bind mounts are more convenient than volumes because you can edit files on the host.

And if you're using Kubernetes, don't use Docker volumes directly — use PersistentVolumeClaims. Docker volumes are a Swarm-era concept; K8s has its own storage abstraction.

// io.thecodeforge — DevOps tutorial

# Stateless app: no volume needed
docker run -d --name stateless nginx:alpine

# CI job: use tmpfs for scratch
docker run --rm --tmpfs /build:size=500M node:18 sh -c "npm ci && npm run build"

# Development: bind mount for live reload
docker run -d -p 3000:3000 -v $(pwd):/app node:18 npm run dev

# K8s: use PVC, not Docker volumes
# apiVersion: v1
# kind: PersistentVolumeClaim
# metadata:
#   name: mypvc
# spec:
#   accessModes:
#     - ReadWriteOnce
#   resources:
#     requests:
#       storage: 1Gi