Redis Write Failures — Default Eviction Policy

Every high-traffic application hits the same wall eventually: the database becomes the bottleneck. A query that takes 40ms feels invisible during development but becomes catastrophic when 10,000 users hit it simultaneously. Twitter, GitHub, Stack Overflow, and Shopify all reached this wall — and Redis is a big part of how they broke through it. It's not an exaggeration to say that understanding Redis is what separates junior developers from engineers who can design systems that actually scale.

Redis (Remote Dictionary Server) solves the read-amplification problem. Most web applications read data far more than they write it — a product page might be read 50,000 times a day but updated once. Hammering your relational database with 50,000 identical queries is wasteful and slow. Redis lets you compute the answer once, store it in memory, and serve all 50,000 requests from there in microseconds. But Redis isn't just a cache — it's a full data structure server that can power rate limiters, leaderboards, pub/sub messaging, session stores, and queues.

By the end of this article you'll understand not just what Redis commands look like, but WHY each data structure exists, WHEN to reach for each one, and how to wire Redis into a real application pattern. You'll also learn the subtle mistakes — wrong expiry strategies, cache stampedes, missing persistence configs — that trip up developers who learned Redis from a cheat sheet instead of from first principles.

What Redis Eviction Policy Actually Does

Redis is an in-memory key-value store where all data lives in RAM. When memory fills up, Redis must decide which keys to remove to make room for new writes — that decision is the eviction policy. By default, Redis uses no eviction (noeviction), meaning writes fail with an OOM error when maxmemory is reached. This is not a bug; it's a deliberate safety mechanism to prevent data loss without explicit configuration.

The eviction policy is set via maxmemory-policy in redis.conf. With noeviction, Redis rejects SET, LPUSH, and other write commands, returning an error to the client. Other policies like allkeys-lru or volatile-ttl evict keys based on usage or TTL. The default choice forces you to acknowledge memory limits — a design that prevents silent data loss but requires proactive capacity planning.

Use noeviction when data integrity is critical and you can predict memory usage — for example, caching session tokens with fixed TTLs. In production, teams often switch to allkeys-lru for general caching, but the default catches engineers off guard when memory spikes cause sudden write failures. Understanding this default is the first step to designing resilient Redis deployments.

Why Redis Lives in RAM and Why That Changes Everything

Traditional databases store data on disk. Disk access — even an NVMe SSD — operates in the microseconds-to-milliseconds range. RAM access operates in nanoseconds. That's not a small difference; it's three orders of magnitude. Redis keeps its entire dataset in memory by default, which is the single most important architectural decision behind its speed.

But speed isn't the only trick. Redis is single-threaded for command execution. That sounds like a limitation until you understand what it eliminates: lock contention. In a multi-threaded database, threads fight over the same rows with locks. Redis sidesteps that fight entirely — one command runs to completion before the next starts. This makes Redis operations atomic by default, which matters enormously for things like incrementing a counter or checking-then-setting a value.

Redis also supports optional persistence. You can tell it to snapshot its RAM contents to disk every N seconds (RDB snapshotting) or to log every write command to an append-only file (AOF). Most production setups use both. This means Redis isn't just a volatile cache — it can survive a restart and recover its data.

The practical takeaway: use Redis for data that is read far more than it's written, where milliseconds matter, and where you can tolerate the data being slightly stale or reproducible if lost.

# --- Step 1: Start the Redis server (run in one terminal) ---
# This launches Redis with default config on port 6379
redis-server

# --- Step 2: Connect with the Redis CLI (run in another terminal) ---
redis-cli

# --- Step 3: Ping the server to confirm it's alive ---
127.0.0.1:6379> PING
# Redis responds with PONG — the simplest health check you'll ever do

# --- Step 4: Store a string value (the most basic operation) ---
# SET key value
# We're caching a user's display name keyed by their user ID
127.0.0.1:6379> SET user:1001:display_name "Alice Nguyen"

# --- Step 5: Retrieve it ---
127.0.0.1:6379> GET user:1001:display_name

# --- Step 6: Store a value with an expiry (TTL = Time To Live) ---
# EX sets expiry in seconds — this key auto-deletes after 300 seconds (5 minutes)
# This is the pattern for caching: store it, let it expire, recompute if missing
127.0.0.1:6379> SET product:42:price "29.99" EX 300

# --- Step 7: Check how many seconds remain before expiry ---
127.0.0.1:6379> TTL product:42:price

# --- Step 8: Check if a key exists without fetching its value ---
127.0.0.1:6379> EXISTS user:1001:display_name

Redis Data Structures — Picking the Right Tool for Each Problem

Redis isn't just a key-value store in the boring sense. It stores five core data types, and choosing the right one is the difference between an elegant solution and a painful hack.

Strings — the default. Good for counters, cached HTML, serialized JSON blobs, and session tokens. The INCR command atomically increments a string-as-integer, making it perfect for rate limiting and hit counters.

Hashes — think of a Hash as a mini dictionary attached to one key. Instead of storing a user as one giant JSON blob, you store their fields separately. This lets you update a single field without fetching and re-serializing the entire object.

Lists — ordered, duplicates allowed. Built on a linked list internally. Ideal for queues (push to the tail, pop from the head) and activity feeds (push new events to the head, trim the list to keep only the last N).

Sets — unordered, unique members. Perfect for tracking unique visitors, tagging systems, or finding common followers between two users with SINTER.

Sorted Sets — the crown jewel. Every member has a floating-point score. Redis keeps members ordered by score automatically. This is how you build leaderboards, priority queues, and range-based queries without a single SQL ORDER BY.

# =========================================================
# STRINGS — atomic counter for API rate limiting
# =========================================================

# Track how many API calls user 2055 has made this minute
# INCR is atomic — safe even with concurrent requests
127.0.0.1:6379> INCR api_calls:user:2055:minute:2024061514
127.0.0.1:6379> INCR api_calls:user:2055:minute:2024061514
127.0.0.1:6379> INCR api_calls:user:2055:minute:2024061514

# Set it to expire at the end of the minute (60 seconds)
127.0.0.1:6379> EXPIRE api_calls:user:2055:minute:2024061514 60

# =========================================================
# HASHES — store a user profile without one giant JSON blob
# =========================================================

# HSET sets one or more fields on a hash key
# Updating just the email later only rewrites that one field
127.0.0.1:6379> HSET user:2055 username "bob_the_dev" email "bob@example.com" plan "pro" login_count 0

# Retrieve a single field — no need to deserialize a full JSON object
127.0.0.1:6379> HGET user:2055 email

# Retrieve all fields at once
127.0.0.1:6379> HGETALL user:2055

# Atomically increment just the login counter
127.0.0.1:6379> HINCRBY user:2055 login_count 1

# =========================================================
# SORTED SETS — real-time game leaderboard
# =========================================================

# ZADD leaderboard_key score member
# Score is the player's points — Redis sorts automatically
127.0.0.1:6379> ZADD game:leaderboard 4200 "player:alice"
127.0.0.1:6379> ZADD game:leaderboard 8750 "player:bob"
127.0.0.1:6379> ZADD game:leaderboard 6100 "player:carol"

# Fetch top 3 players, highest score first (WITHSCORES shows the score)
# ZREVRANGE = reverse order = highest to lowest
127.0.0.1:6379> ZREVRANGE game:leaderboard 0 2 WITHSCORES

# Get a specific player's rank (0-indexed, 0 = top)
127.0.0.1:6379> ZREVRANK game:leaderboard "player:alice"

# =========================================================
# LISTS — lightweight task queue
# =========================================================

# LPUSH adds to the LEFT (head) of the list
# Workers will RPOP from the RIGHT (tail) — FIFO queue
127.0.0.1:6379> RPUSH email_queue "{\"to\":\"alice@example.com\",\"subject\":\"Welcome\"}"
127.0.0.1:6379> RPUSH email_queue "{\"to\":\"bob@example.com\",\"subject\":\"Reset\"}"

# BLPOP = blocking pop — worker waits up to 5 seconds for a job
# This is more efficient than polling in a loop
127.0.0.1:6379> BLPOP email_queue 5

The Cache-Aside Pattern — Wiring Redis Into a Real Application

Knowing Redis commands is one thing. Knowing how to integrate Redis into your application code without creating subtle bugs is another. The most widely-used pattern is Cache-Aside (also called Lazy Loading). The logic is elegantly simple: when your app needs data, check Redis first. If it's there (a cache hit), return it immediately. If it's not (a cache miss), fetch it from the database, store it in Redis with a TTL, then return it. Redis never gets data pushed to it — your application pulls it through.

This pattern is powerful because it's self-healing. If Redis goes down and loses all its data, your app degrades gracefully — everything just goes to the database until Redis is warm again. The cache populates itself organically based on what users actually request, not what you predict they'll request.

The critical detail most tutorials skip: always set a TTL. Without one, your cache grows forever and you'll eventually run out of RAM. More importantly, stale data lives forever. If a product's price changes in your database but the Redis entry never expires, customers see wrong prices indefinitely. Your TTL is your freshness guarantee.

The code below shows this pattern implemented in Python with the redis-py library — the same library used by Instagram and Pinterest in production.

import redis
import json
import time

# --- Connect to Redis ---
# decode_responses=True means Redis returns strings instead of bytes
redis_client = redis.Redis(
    host="localhost",
    port=6379,
    db=0,
    decode_responses=True
)

# --- Simulated database fetch (replace with your real DB query) ---
def fetch_product_from_database(product_id: int) -> dict:
    """
    Simulates a slow database query.
    In production this would be: cursor.execute('SELECT * FROM products WHERE id = %s', [product_id])
    """
    print(f"[DB] Querying database for product {product_id}...")
    time.sleep(0.05)  # simulate 50ms DB query latency
    return {
        "id": product_id,
        "name": "Mechanical Keyboard TKL",
        "price": 129.99,
        "stock": 42
    }

def get_product(product_id: int) -> dict:
    """
    Cache-Aside Pattern implementation.
    Always check Redis first. Fall back to DB on miss. Always set a TTL.
    """
    cache_key = f"product:{product_id}"  # namespaced key following the colon convention
    cache_ttl_seconds = 300              # cache is valid for 5 minutes

    # --- Step 1: Try the cache first ---
    cached_value = redis_client.get(cache_key)

    if cached_value is not None:
        # Cache HIT — data found in Redis, no DB query needed
        print(f"[CACHE] Hit for key '{cache_key}'")
        return json.loads(cached_value)  # deserialize from JSON string back to dict

    # --- Step 2: Cache MISS — go to the database ---
    print(f"[CACHE] Miss for key '{cache_key}'")
    product_data = fetch_product_from_database(product_id)

    # --- Step 3: Populate the cache for next time ---
    # json.dumps serializes the dict to a JSON string for storage
    # ex=cache_ttl_seconds ensures the key auto-expires — NEVER skip this
    redis_client.set(
        cache_key,
        json.dumps(product_data),
        ex=cache_ttl_seconds
    )
    print(f"[CACHE] Stored '{cache_key}' with TTL={cache_ttl_seconds}s")

    return product_data

def invalidate_product_cache(product_id: int):
    """
    Call this whenever a product is updated in the database.
    Removing the key forces the next request to re-fetch fresh data.
    """
    cache_key = f"product:{product_id}"
    deleted_count = redis_client.delete(cache_key)
    if deleted_count > 0:
        print(f"[CACHE] Invalidated key '{cache_key}'")
    else:
        print(f"[CACHE] Key '{cache_key}' wasn't in cache — nothing to invalidate")

# --- Demo ---
if __name__ == "__main__":
    print("=== First request — cold cache ===")
    product = get_product(product_id=7)
    print(f"Result: {product}\n")

    print("=== Second request — warm cache ===")
    product = get_product(product_id=7)
    print(f"Result: {product}\n")

    print("=== Simulating a product update ===")
    invalidate_product_cache(product_id=7)

    print("\n=== Third request — cache was invalidated ===")
    product = get_product(product_id=7)
    print(f"Result: {product}")

Redis Expiry, Eviction and Why Your Cache Will Betray You Without Them

TTLs are your first line of defense against stale data. But what happens when Redis runs out of memory before any keys expire? This is where eviction policies come in, and most developers don't think about them until Redis starts refusing writes in production — which is a very bad day.

Redis has several eviction policies configured via maxmemory-policy in your redis.conf. The default policy is noeviction — Redis refuses new writes when full. That sounds safe but it means your application starts throwing errors. For a cache, you almost always want allkeys-lru (evict the least recently used key across all keys) or volatile-lru (evict the least recently used key that has a TTL set).

There's also the cache stampede problem — also called the thundering herd. Imagine 500 concurrent users all request the same popular product page. The cache entry expires at the exact same moment. All 500 requests find a cache miss simultaneously and all fire a database query at once. Your database gets hammered with 500 identical queries in the same millisecond. The fix is probabilistic early expiration or using a mutex lock in your cache-miss path so only one request rebuilds the cache while others wait.

The rule of thumb: if your cache powers any page that gets high traffic, you need to think about stampedes. If your cache serves data with truly random access patterns, you probably don't.

# =========================================================
# CONFIGURING EVICTION POLICY (redis.conf or at runtime)
# =========================================================

# Set max memory to 256MB — Redis will start evicting when this is reached
127.0.0.1:6379> CONFIG SET maxmemory 268435456

# allkeys-lru = evict least-recently-used key from the ENTIRE keyspace
# Best default for a pure cache where all keys have roughly equal value
127.0.0.1:6379> CONFIG SET maxmemory-policy allkeys-lru

# Confirm the config was applied
127.0.0.1:6379> CONFIG GET maxmemory-policy

# =========================================================
# TTL COMMANDS — managing key lifetime
# =========================================================

# SET with expiry in seconds
127.0.0.1:6379> SET session:user:8821 "eyJhbGciOiJIUzI1NiJ9" EX 3600

# SET with expiry in milliseconds (for sub-second precision)
127.0.0.1:6379> SET rate_check:ip:192.168.1.1 "1" PX 60000

# Check TTL in seconds (-1 = no expiry, -2 = key doesn't exist)
127.0.0.1:6379> TTL session:user:8821

# Check TTL in milliseconds (more precise)
127.0.0.1:6379> PTTL rate_check:ip:192.168.1.1

# PERSIST removes the TTL — key lives forever (use with caution)
127.0.0.1:6379> PERSIST session:user:8821
127.0.0.1:6379> TTL session:user:8821

# =========================================================
# MUTEX PATTERN — prevent cache stampedes
# NX = only set if key does NOT exist (atomic check-and-set)
# This is a distributed lock: only the first caller wins
# =========================================================

# First request tries to acquire the rebuild lock (TTL=10s to auto-release)
127.0.0.1:6379> SET lock:product:7:rebuild "1" NX EX 10

# Second concurrent request tries the same — gets nil (lock is taken)
127.0.0.1:6379> SET lock:product:7:rebuild "1" NX EX 10

# After the first request rebuilds the cache and releases the lock:
127.0.0.1:6379> DEL lock:product:7:rebuild

# =========================================================
# CHECK MEMORY USAGE
# =========================================================

# See overall memory stats
127.0.0.1:6379> INFO memory

# See exactly how much RAM one key is using (in bytes)
127.0.0.1:6379> MEMORY USAGE session:user:8821

Redis Eviction Policies — Technical Comparison Table

Choosing the right eviction policy is one of the most consequential Redis decisions you'll make in production. Each policy has a specific use case, and using the wrong one can silently corrupt your data or crash your application. Below is a technical comparison of all eight eviction policies available in Redis 7.x.

How to choose: - Pure cache: allkeys-lru or allkeys-lfu (if access pattern is skewed). - Session store with TTLs: volatile-ttl (keys with short TTL evicted first – likely stale anyway) or volatile-lru. - Permanent data only: noeviction is acceptable if you never hit the memory limit, but always monitor maxmemory. - Random access: allkeys-random is simple and predictable.

Algorithm internals: Redis LRU/LFU are approximations, not exact. They use a sample pool of keys (default 5) to pick the one to evict. This is O(1) with low overhead. You can tune the sample size via maxmemory-samples in redis.conf. Larger samples improve eviction quality but use more CPU.

Production checklist: 1. Never rely on the default noeviction for any cache workload. 2. Set maxmemory based on available RAM and headroom for other processes. 3. Monitor evicted_keys in INFO stats – if it's >0, your cache is under memory pressure. 4. Combine eviction with TTLs to reduce pressure on LRU/LFU algorithms.

# Check current eviction policy and stats
127.0.0.1:6379> CONFIG GET maxmemory-policy
127.0.0.1:6379> INFO stats | grep evicted_keys
127.0.0.1:6379> CONFIG GET maxmemory-samples

# Change policy at runtime
127.0.0.1:6379> CONFIG SET maxmemory-policy allkeys-lfu
127.0.0.1:6379> CONFIG SET maxmemory-samples 10

# View current eviction stats over the last 5 seconds
127.0.0.1:6379> INFO stats | grep -E "evicted|expired"

# Test which key would be evicted next (Redis 7+)
127.0.0.1:6379> MEMORY DOCTOR

Redis Persistence — RDB vs AOF and Production Trade-offs

By default, Redis stores everything in RAM. If the server restarts, all data is lost. For a cache, that's acceptable. But many teams use Redis as a session store, a rate-limiter state store, or even a primary database for high-frequency writes. In those cases, losing data on restart is catastrophic.

Redis offers two persistence mechanisms:

RDB (Redis Database) — periodic snapshots of the entire dataset to disk. You configure how often (e.g., save 900 1 means if at least 1 key changed in 900 seconds, save). RDB files are compact and great for backups/disaster recovery. The downside: you lose data between snapshots. A crash 10 minutes before the next snapshot loses 10 minutes of writes.

AOF (Append Only File) — logs every write command to an append-only file. You can replay the file on restart to reconstruct the dataset. AOF gives you finer granularity: you can configure appendfsync everysec (lose at most 1 second of writes) or always (every write forces disk sync, near-zero data loss but 30-50% slower writes).

Most production setups use both. Redis supports a combined mode: RDB for fast restores, AOF for durability. When both are enabled, Redis loads the AOF on restart because it's more complete.

The choice matters for your data loss tolerance. Session stores need AOF everysec. Cache layers don't need persistence at all. A leaderboard that can rebuild from database can afford RDB-only. Match the persistence config to your data's criticality.

# =========================================================
# CONFIGURING PERSISTENCE (redis.conf)
# =========================================================

# --- RDB Snapshotting ---
# Format: save <seconds> <changes>
# Trigger: at least 1 change in 900s (15 min) OR 10 in 300s (5 min) OR 10000 in 60s
save 900 1
save 300 10
save 60 10000

# RDB file location and name
dbfilename dump.rdb
dir /var/lib/redis/

# Compress the RDB file (yes/no)
rdbcompression yes

# --- AOF Persistence ---
# Enable AOF (yes/no)
appendonly yes

# AOF file name
appendfilename "appendonly.aof"

# Sync policy:
# always = sync every write (safest, slowest)
# everysec = sync once per second (default, best trade-off)
# no = let OS decide (fastest, least safe)
appendfsync everysec

# --- AOF Rewrite ---
# Auto-rewrite AOF when it grows 100% in size (auto-aof-rewrite-percentage 100)
# and reaches at least 64MB (auto-aof-rewrite-min-size 64mb)
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# --- Check current config ---
# From redis-cli
127.0.0.1:6379> CONFIG GET save
127.0.0.1:6379> CONFIG GET appendonly
127.0.0.1:6379> CONFIG GET appendfsync

# --- Force an RDB save immediately ---
127.0.0.1:6379> SAVE

# --- Force an AOF rewrite ---
127.0.0.1:6379> BGREWRITEAOF

# --- Check persistence stats ---
127.0.0.1:6379> INFO persistence

RDB vs AOF — Decision Matrix

Choosing between RDB and AOF (or both) depends on your data loss tolerance, write throughput, and recovery speed requirements. This decision matrix helps you pick the right combination for your production workload.

Decision rules: 1. Cache only → No persistence. If Redis crashes, the cache rebuilds from the database. 2. Session store → AOF everysec. Losing sessions forces mass logouts; 1-second loss is acceptable. 3. Primary data store → Both. RDB for fast recovery (if AOF is corrupted) and AOF for durability. 4. Rate limiter / metadata → AOF everysec or RDB-only depending on whether state can be rebuilt. 5. Cross-version upgrade → RDB-only redises persist across versions; AOF may require rewriting.

Production recommendation: Use RDB snapshots for backup and AOF everysec for crash recovery. Configure AOF auto-rewrite (auto-aof-rewrite-percentage 100, auto-aof-rewrite-min-size 64mb) to keep AOF files manageable. Test your recovery procedure quarterly by simulating a server restart.

# Enable both persistence methods
127.0.0.1:6379> CONFIG SET save "900 1 300 10 60 10000"
127.0.0.1:6379> CONFIG SET appendonly yes
127.0.0.1:6379> CONFIG SET appendfsync everysec
127.0.0.1:6379> CONFIG SET auto-aof-rewrite-percentage 100
127.0.0.1:6379> CONFIG SET auto-aof-rewrite-min-size 64mb

# Verify
127.0.0.1:6379> CONFIG GET appendfsync
127.0.0.1:6379> INFO persistence | grep aof_current_size

# Simulate a clean shutdown to flush all data
127.0.0.1:6379> SHUTDOWN SAVE

# After restart, check which file Redis loaded
# Look for "DB loaded from disk" or "DB loaded from AOF" in Redis log
# Typically: tail -20 /var/log/redis/redis-server.log
# Expect line: "DB loaded from append only file: AOF enabled"

Redis Persistence Architecture — Visual Diagram

The diagram illustrates two separate paths for persistence. On the left (solid arrows), every write command is buffered and, depending on appendfsync, flushed to the AOF file on disk. On the right (dashed arrows), a background process BGSAVE forks and writes a snapshot (RDB file) at configured intervals. During restart, Redis checks for persistence files: AOF takes precedence if both exist. If only RDB exists, it loads the snapshot; if neither, the dataset starts empty.

Key architectural details: - RDB uses copy-on-write: the fork creates a child process that snapshots the memory at that instant. The parent continues serving requests, modified pages are copied via COW. This can spike memory usage (if many pages are modified during save). - AOF rewrite also forks: the child reads the existing AOF and creates a compact version in a temp file, then swaps it atomically. - Both mechanisms rely on fsync() to ensure data is actually on disk. appendfsync everysec issues fsync() once per second in a background thread; appendfsync always calls fsync() for every write, synchronously blocking the main thread.

What this means in practice: - If you enable both RDB and AOF, restart time is determined by AOF replay speed (linear with size). Large AOF files (>5 GB) can take seconds to load. - If you care about startup time and have AOF enabled, schedule regular BGREWRITEAOF to keep the file small. - For read-heavy workloads, RDB alone may be sufficient because restarts are fast and data loss is acceptable.

Redis Cluster — Architectural Overview

Redis Cluster provides automatic sharding and high availability without needing a separate proxy. It's the production architecture for any dataset exceeding a single server's RAM or any workload demanding horizontal scalability. Understanding its architecture is essential for engineers building large-scale caching and real-time data platforms.

Key Components: - Nodes: Each node in a cluster is a regular Redis server running in cluster mode (cluster-enabled yes). - Hash Slots: The keyspace is divided into 16,384 hash slots (hash function: CRC16(key) mod 16384). Each node is responsible for a subset of slots. - Cluster Bus: A dedicated TCP channel (port + 10000) for inter-node communication: heartbeat, gossip, failover. - Configuration: Every node stores the cluster configuration (nodes.conf) with the slot assignment and the view of all nodes. - Replicas: Each master node can have one or more replicas that replicate its data. If a master fails, a replica is promoted.

How Data Is Distributed: When a client sends a command for a specific key, the Redis client library (e.g., redis-py-cluster) computes the hash slot and sends the command directly to the node responsible for that slot. If the client sends to the wrong node, that node returns a -MOVED redirect error with the correct node's address. Smart clients cache the slot mapping to avoid redirects.

Resharding: Adding or removing nodes involves moving hash slots between nodes. This is done online via redis-cli --cluster reshard. During migration, slots are source nodes mark as migrating and target nodes as importing. Keys are migrated in batches using MIGRATE command. The cluster remains available during resharding.

Failover: Master failure detection is based on a quorum of nodes marking the master as PFAIL (possibly failing), then FAIL if a majority agree. A replica then triggers an election using the cluster bus. The winning replica increases its currentEpoch and becomes master. Automatic failover requires that the majority of masters are reachable.

Limitations: - Multi-key operations (e.g., MGET, transactions, Lua scripts) only work if all keys hash to the same slot. Use hash tags {key} to force keys into the same slot. - No support for multiple databases (only DB 0). - Minimum 3 master nodes recommended; for high availability, 3 masters + 3 replicas in different racks/availability zones. - Network latency between nodes adds overhead for cluster bus and replication.

When to use Redis Cluster: - Dataset > 10 GB (or whatever fits in one node's RAM). - Need for automatic failover and high availability. - Write throughput exceeds what a single Redis instance can handle. - You need linear scalability by adding nodes.

When not to use: - Your dataset fits in one node and you don't need high availability – a single instance with Sentinel is simpler. - You rely heavily on multi-key operations across different keys. - Your workload is predominantly read-heavy with a small dataset – single instance is faster.

# --- Step 1: Configure each node in cluster mode (add to redis.conf) ---
# On each of 3 servers (node1, node2, node3):
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
port 6379

# --- Step 2: Start each Redis instance ---
redis-server /path/to/redis.conf &

# --- Step 3: Create cluster (run from one node) ---
# Replace IPs with actual server IPs. Use --cluster-replicas 1 for one replica per master
redis-cli --cluster create \
  192.168.1.10:6379 \
  192.168.1.11:6379 \
  192.168.1.12:6379 \
  --cluster-replicas 1

# --- Step 4: Check cluster status ---
127.0.0.1:6379> CLUSTER INFO
127.0.0.1:6379> CLUSTER NODES
127.0.0.1:6379> CLUSTER SLOTS

# --- Step 5: Test key distribution ---
# Connect to any node and set a key; simulate redirected client
127.0.0.1:6379> SET user:1001 "Alice"
# If sent to wrong node, responds:
# -MOVED 7629 192.168.1.11:6379

# --- Step 6: Reshard example (migrate 100 slots from node 1 to node 4) ---
redis-cli --cluster reshard 192.168.1.10:6379 --cluster-from <node-id-1> --cluster-to <node-id-4> --cluster-slots 100

Cache Hit vs Cache Miss — The Two Paths That Decide Your Latency

Every Redis request takes one of two paths. Cache hit means data was in RAM. Cache miss means you're about to pay the disk tax. Simple, right? Yet I've seen teams flame out because they only tested the happy path.

A cache hit returns data in sub-millisecond time. The application gets its response, the database never sees the request. That's the whole point of Redis.

A cache miss is the expensive fallback. Redis returns nil, the application queries the primary database, writes the result back to Redis for next time, then serves the response. That's at least one network round trip plus a disk read. In high traffic, a sudden wave of misses — from a cache flush or deployment — can take down your database.

Design for cache misses. Set proper expiry. Pre-warm caches after restart. Your database doesn't have a second chance.

// io.thecodeforge — database tutorial

-- Simulating a cache miss + write-back in PostgreSQL
-- This is NOT Redis itself, it shows the app fallback logic

BEGIN;

-- Step 1: Check Redis (pseudo) — SELECT from cache table as stand-in
SELECT value FROM redis_cache WHERE key = 'user:profile:47291';

-- If returned 0 rows (CACHE MISS) → hit primary DB
SELECT id, username, email, preferences_json 
FROM users 
WHERE id = 47291;

-- Step 2: Write result back to cache for next request
INSERT INTO redis_cache (key, value, expiry_epoch)
VALUES ('user:profile:47291', '{"id":47291,"username":"jane_dev","prefs":{"theme":"dark"}}', extract(epoch from now()) + 3600)
ON CONFLICT (key) DO UPDATE 
SET value = EXCLUDED.value, expiry_epoch = EXCLUDED.expiry_epoch;

-- Step 3: Return to client
SELECT 'SUCCESS' AS response;

COMMIT;

Real-World Redis — Where the Big Shops Actually Use It

Competitor pages list 'Netflix uses Redis for caching' like it's a revelation. Let's get specific about where Redis earns its keep in production.

E-commerce — product catalog caches. Amazon doesn't query DynamoDB for every page load. They push the entire category view into Redis hashes. Product ID as key, all attributes as fields. Fetch in one round trip. Fresh inventory at scale.

Real-time gaming leaderboards — sorted sets with scores. Every player action updates their score via ZINCRBY. Top 100 is ZREVRANGE with LIMIT. No SQL JOINs. No page refreshes. This is why your mobile game updates ranks instantly.

Rate limiting — INCR with EXPIRE. User hits an endpoint, you INCR a key like ratelimit:user:47291:minute. Set TTL to 60 seconds. If value > threshold, reject. Atomic. Fast. No database write. Instagram uses this pattern for API throttling.

Session store — EXPIRE handles user logout automatically. No cron jobs to clean stale sessions. No table scans.

// io.thecodeforge — database tutorial

-- Rate limit check pattern (pseudo-Redis logic in PostgreSQL)
-- NOT actual Redis, but shows the atomic increment structure

CREATE OR REPLACE FUNCTION rate_limit_user(p_user_id INT)
RETURNS BOOLEAN AS $$
DECLARE
  v_count INT;
  v_max_requests INT := 100;  -- per minute
BEGIN
  -- Simulate Redis INCR + EXPIRE
  INSERT INTO rate_limiter (user_id, counter, window_start)
  VALUES (p_user_id, 1, extract(epoch from now())::bigint / 60)
  ON CONFLICT (user_id, window_start) DO UPDATE
  SET counter = rate_limiter.counter + 1
  RETURNING counter INTO v_count;
  
  -- Check threshold
  IF v_count > v_max_requests THEN
    RETURN FALSE;  -- BLOCK
  END IF;
  
  RETURN TRUE;  -- ALLOW
END;
$$ LANGUAGE plpgsql;

What Makes Redis Fast — The Three Pillars (Spoiler: Not Just RAM)

Everyone says 'Redis is fast because it's in-memory.' True, but lazy. Plenty of in-memory databases are slow. Redis has three architectural decisions that matter more than RAM.

Single-threaded event loop. No locks. No context switching between threads. One queue processes all commands sequentially. This means no race conditions on simple operations, no mutex overhead. Write a complex Lua script? It blocks everything else. Don't write slow Lua scripts.

Non-blocking I/O with epoll/kqueue. Redis doesn't spin waiting for network data. It registers file descriptors and gets notified when data arrives. This is why a single Redis instance handles 100K+ ops per second on modest hardware.

Data structures tuned for cache lines. Redis strings, lists, hashes are designed to fit in CPU L2 cache. Compare that to PostgreSQL which is optimised for disk pages. Redis data is already hot in memory and laid out for minimal pointer chasing.

Combine these: single-threaded + event-driven + cache-optimised structures. That's the real answer. Not just 'it's in RAM'.

// io.thecodeforge — database tutorial

-- Benchmark simulation comparing in-memory vs disk latency
-- Shows the orders of magnitude difference

WITH latencies AS (
  SELECT 'L1 Cache' AS location, 1 AS nanoseconds_per_op, 1 AS relative_speed
  UNION ALL SELECT 'L2 Cache', 7, 7
  UNION ALL SELECT 'RAM', 100, 100
  UNION ALL SELECT 'SSD (NVMe)', 10000, 10000
  UNION ALL SELECT 'HDD (Spinning)', 5000000, 5000000  -- 5 ms
)
SELECT 
  location,
  nanoseconds_per_op,
  CASE 
    WHEN relative_speed <= 100 THEN 'HOT — CPU Cache'
    WHEN relative_speed <= 10000 THEN 'WARM — RAM'
    ELSE 'COLD — Disk'
  END AS redis_temperature
FROM latencies
ORDER BY relative_speed;

Working With Numbers — Redis Isn't Just for Strings

Redis treats numbers as strings internally but provides atomic operations for integer arithmetic. When you store a numeric value, Redis sees a string that can be incremented or decremented using commands like INCR, DECR, INCRBY, and DECRBY. These operations are atomic—no race conditions, no lost updates. This makes Redis an excellent choice for real-time counters, rate limiters, and leaderboards. The WHY: atomicity eliminates the need for locks or transactions in single-instance scenarios. If you try to increment a non-numeric value (like a word), Redis returns an error. The error message is clear: 'ERR value is not an integer or out of range'. Always validate your data type before performing arithmetic. Redis stores integers as signed 64-bit values, so overflow will wrap around silently—watch for that in high-throughput counters.

// io.thecodeforge — database tutorial

// Atomic increment and decrement operations
SET views:post:42 0
INCR views:post:42
// Returns (integer) 1
DECR views:post:42
// Returns (integer) 0
INCRBY views:post:42 10
// Returns (integer) 10
SET badkey "hello"
INCR badkey
// Returns (error) ERR value is not an integer or out of range

Saving and Retrieving Key-Value Pairs — The Foundation of Every Redis App

Redis stores data as key-value pairs. The key is always a string, and the value can be one of many data types: string, list, set, hash, or sorted set. The most basic operations are SET and GET. SET associates a key with a value; GET retrieves it. If the key doesn't exist, GET returns nil (null in most clients). The WHY: Redis uses keyspace as a flat dictionary—no schema, no tables, no joins. This zero-overhead model delivers microsecond latency for single-key lookups. When you SET a key that already exists, the old value is silently overwritten. To avoid accidental overwrites, use SETNX (set if not exists) or check for nil before writing. Namespacing keys with colons (like 'user:1000:name') is a standard practice—Redis handles long keys fine, but shorter keys mean less memory overhead. TTL expiry is often set at write time to control cache lifetimes.

// io.thecodeforge — database tutorial

// Basic SET and GET operations
SET user:1000:name "Alice"
GET user:1000:name
// Returns "Alice"
GET user:9999:name
// Returns (nil)
SETNX user:1000:name "Bob"
// Returns (integer) 0 — key already exists, not set
SETNX user:1001:name "Bob"
// Returns (integer) 1 — new key set successfully