Redis Data Structures — Memory Eviction from Missing TTL

You’ve used Redis for caching, sure. But its value lives in the data structures themselves—strings, hashes, lists, sets, sorted sets, bitmaps, and HyperLogLog. Pick the wrong one or forget TTL discipline, and you’ll paint yourself into a corner with memory bloat or O(n) bottlenecks. This article walks each structure, its real-world job, and the command complexity you need to know before your next deploy.

Why Redis Data Structures Need TTL Discipline

Redis data structures are in-memory key-value stores where each key maps to one of several native types: strings, hashes, lists, sets, sorted sets, bitmaps, hyperloglogs, streams, or geospatial indexes. The core mechanic is that every operation runs in O(1) or O(log n) time, but memory is finite. When no TTL (time-to-live) is set, keys persist indefinitely, and Redis must evict data under the configured maxmemory policy once RAM is exhausted. This eviction is not a bug—it's a deliberate trade-off between availability and data retention.

In practice, Redis uses an approximate LRU (or LFU, TTL-based, random) algorithm to select keys for eviction. The default policy is noeviction, which causes writes to fail with an OOM error. More common policies like allkeys-lru or volatile-lru evict keys without TTL or only those with TTL, respectively. The key property: eviction happens at write time, not proactively. A sudden write spike can trigger mass eviction, removing data you assumed was safe. Without TTLs, all keys are eligible under allkeys-* policies, making your cache behavior unpredictable.

Use this knowledge to design for eviction. In caching layers, always set TTLs to bound memory usage and control eviction scope. For session stores or rate limiters, use volatile-* policies so only expirable keys are evicted. In real systems, a missing TTL on a hot key can cause a cascade: the key gets evicted, the next request recomputes it (expensive), and the surge of recomputation overloads the backend. TTLs are not optional—they are the primary lever for predictable memory management.

Strings — Redis's Swiss Army Knife (and Why It's More Than Just Text)

The String type is the most deceptively simple structure in Redis. It stores a single value — bytes, really — up to 512 MB. That value can be plain text, a serialized JSON blob, a binary image, or an integer. When the value is an integer, Redis lets you increment and decrement it atomically with INCR and DECR, which is the key insight most beginners miss.

Atomic means no race condition. If two servers both call INCR on the same key at the exact same millisecond, Redis executes them sequentially — you get 1, then 2, never 1 and 1. This makes Strings the go-to choice for rate limiting and counting: page views, API calls, login attempts.

The second superpower is the SET options. You can set a value with an expiry in one atomic command using SET key value EX seconds. This is how session tokens work in Redis — store the token as the key, the user ID as the value, and attach a TTL. No separate expiry-management logic needed.

When should you NOT use a String? When your data has multiple fields. Storing a JSON blob like '{"name":"Alice","age":30}' as a String forces you to deserialize the whole thing to update one field. That's exactly when you want a Hash.

-- Rate limiting: allow max 5 requests per user per 60 seconds
-- The key encodes both the user ID and the current time window

-- User 42 makes their first request in this window
SET rate_limit:user:42:window:1700000000 0 EX 60 NX
-- NX means 'only set if key does NOT exist'
-- EX 60 means expire in 60 seconds automatically
-- This gives us a self-cleaning counter

-- Now increment on each request
INCR rate_limit:user:42:window:1700000000
-- Returns: (integer) 1

INCR rate_limit:user:42:window:1700000000
-- Returns: (integer) 2

-- Check value before allowing the request
GET rate_limit:user:42:window:1700000000
-- Returns: "2"

-- Simulate hitting the limit
INCR rate_limit:user:42:window:1700000000
INCR rate_limit:user:42:window:1700000000
INCR rate_limit:user:42:window:1700000000
-- Returns: 5 — at this point your app logic blocks the request

-- Check remaining TTL (useful to tell user when they can retry)
TTL rate_limit:user:42:window:1700000000
-- Returns: (integer) 47  (13 seconds have passed)

Command Complexity Reference Table

Understanding the time complexity of Redis commands is essential for writing efficient production code. Redis is single-threaded — a slow O(N) command blocks all other operations. The table below summarizes the complexity of common commands across data structures. Use it to decide whether your access pattern is safe at scale.

For Strings and Hashes, most operations are O(1). Lists have O(1) push/pop from ends but O(N) for index access. Sets and Sorted Sets have O(1) for membership, but set operations are O(N + M). Sorted Set range queries are O(log N + result set). Streams add and read are O(1) per entry, but XRANGE and XREAD with large ranges can be O(N).

Production note: Always test with your actual data size. A command that is O(log N) on a million-element sorted set (∼20 steps) may be acceptable, but O(N) on a stream with a million entries (a million steps) can block Redis for seconds.

-- Quick commands to estimate your data size
-- Check list length (O(1))
LLEN mylist

-- Check set cardinality (O(1))
SCARD myset

-- Check sorted set size (O(1))
ZCARD myzset

-- Fetch top 10 from sorted set (O(log N) + 10)
ZREVRANGE myzset 0 9 WITHSCORES

-- Check stream length (O(1))
XLEN mystream

-- Generic memory usage of a key (O(N) on the data)
MEMORY USAGE mykey

Data Structure Decision Tree

Choosing the right Redis data structure is the most important performance decision you'll make. The decision tree below guides you step by step from your application requirement to the best Redis data type. Follow the arrows based on your access patterns — ordering, uniqueness, field-level updates — and you'll avoid the common trap of using Strings for everything.

The tree is designed for production decisions: it considers not just functionality but memory efficiency (ziplist encoding in Hashes, bit-level storage in Bitmaps) and blocking behavior (avoid O(N) commands). Use it when you're designing a new feature or refactoring a slow Redis interaction.

Below the diagram, we've included a concrete example: if you need a cache that tracks 'which users have seen this notification' and also produces a feed of recent notifications, the tree tells you to use a Set for the seen-tracking (SISMEMBER) and a List with LTRIM for the feed. If you later need to sort the feed by time, the tree would suggest a Sorted Set with timestamps as scores.

-- Example: Notification system following the decision tree
-- Step 1: Need unique 'seen' flags per user? → Set
SADD seen:user:42 "notif_789"

-- Step 2: Need an ordered feed of recent notifications? → List with trim
LPUSH feed:global "notif_789:User Alice posted"
LTRIM feed:global 0 99

-- Step 3: Need to know who has seen a notification quickly? → SISMEMBER
SISMEMBER seen:user:42 "notif_789"
-- Returns 1

-- Alternatively, if the feed needed timestamps for sorting:
-- Use a Sorted Set with Unix time as score
-- ZADD feed:global 1700000000 "notif_789"
-- ZREVRANGEBYSCORE feed:global +inf -inf WITHSCORES LIMIT 0 10

Hashes and Lists — Modeling Objects and Building Queues

A Hash stores a map of field-value pairs under a single key. Think of it as a lightweight row in a database — one Redis key holds an entire user profile, product record, or config object. The power is granular updates: HSET user:99 email 'new@example.com' updates one field without touching the rest. No deserialization, no re-serialization, no wasted bandwidth.

Hashes are also memory-efficient. When a Hash has fewer than 128 fields and each value is under 64 bytes, Redis internally uses a compact encoding called a ziplist (listpack in newer versions) that's significantly smaller than storing each field as a separate String key. This is a free optimization you get just by modelling your data correctly.

Lists are an ordered sequence of strings, implemented as a doubly-linked list. Elements are pushed and popped from both ends in O(1) time. This makes Lists the natural fit for job queues: producers LPUSH work onto the left end, consumers BRPOP block-wait on the right end. The 'B' in BRPOP is crucial — it means blocking. The consumer sits idle, consuming zero CPU, until work arrives. No polling loop needed.

Lists also work as activity feeds. Push the latest N events with LPUSH and trim the list to a fixed length with LTRIM so it never grows unbounded. Combined, LPUSH and LTRIM on every write give you a capped, real-time feed in two commands.

-- ═══════════════════════════════
-- HASH: Storing a user profile
-- ═══════════════════════════════

-- Set multiple fields at once on first creation
HMSET user:99 \
  username    "alice_dev" \
  email       "alice@example.com" \
  plan        "pro" \
  login_count 0

-- Retrieve the whole profile
HGETALL user:99
-- Returns all fields and values as a flat list

-- Update just the plan without touching other fields
HSET user:99 plan "enterprise"

-- Atomically increment login_count (same as INCR but for Hash fields)
HINCRBY user:99 login_count 1
-- Returns: (integer) 1

-- Only fetch the two fields you actually need (saves bandwidth)
HMGET user:99 username plan
-- Returns: 1) "alice_dev"  2) "enterprise"


-- ═══════════════════════════════
-- LIST: Capped activity feed
-- ═══════════════════════════════

-- Push a new activity event for user 99 to the front of the list
LPUSH activity_feed:user:99 "logged_in"
LPUSH activity_feed:user:99 "viewed_dashboard"
LPUSH activity_feed:user:99 "exported_report"

-- Trim to the 10 most recent events — run this after every LPUSH
-- Index 0 = newest, index 9 = 10th newest, -1 = last element
LTRIM activity_feed:user:99 0 9

-- Read the 5 most recent events (newest first)
LRANGE activity_feed:user:99 0 4

-- Job queue: producer pushes, consumer blocks until work arrives
LPUSH email_queue "send_welcome:user:99"

-- On the worker process (blocks for up to 30 seconds)
BRPOP email_queue 30
-- Returns: 1) "email_queue"  2) "send_welcome:user:99"

Sets and Sorted Sets — Unique Collections and Real-Time Leaderboards

A Set is an unordered collection of unique strings. 'Unique' is the whole point — Redis enforces it automatically, so you never write deduplication logic yourself. The classic use cases are tagging systems, tracking which users have seen a notification, and social graph relationships like followers.

The feature that makes Sets genuinely powerful beyond simple uniqueness is set algebra: SINTER (intersection), SUNION (union), and SDIFF (difference) let you compute 'users who follow both Alice and Bob', 'all tags on either post', or 'users who bought A but not B' in a single command. Doing that computation in your application layer means pulling thousands of IDs over the network first. Doing it in Redis means only the result travels over the wire.

Sorted Sets (ZSets) are the crown jewel of Redis. Every member gets a floating-point score, and Redis keeps members ordered by that score at all times. Insertion, removal, and score updates are O(log N). Range queries — 'give me the top 10 players', 'give me everyone with a score between 1000 and 2000' — are also O(log N + result size). This is the data structure behind every real-time leaderboard you've ever used.

The score can represent anything orderable: timestamps (for time-sorted feeds), relevance scores (for search ranking), or geographic distances (for proximity search via geohashing). If you need ordered access by any numeric dimension, a Sorted Set is almost always the answer.

-- ═══════════════════════════════
-- SET: Social follow graph
-- ═══════════════════════════════

-- Store who user:alice follows
SADD following:alice  "bob" "carol" "dave"
SADD following:bob    "alice" "carol" "eve"

-- Store alice's followers
SADD followers:alice  "bob" "dave"

-- Who do alice AND bob both follow? (mutual interests)
SINTER following:alice following:bob
-- Returns: 1) "carol"  (both follow carol)

-- Everyone either alice or bob follows (combined feed sources)
SUNION following:alice following:bob

-- Mark notification 789 as seen by user alice (deduplication)
SADD seen_notifications:alice "789"

-- Has alice seen it? O(1) lookup
SISMEMBER seen_notifications:alice "789"
-- Returns: (integer) 1  (yes)

SISMEMBER seen_notifications:alice "999"
-- Returns: (integer) 0  (no)


-- ═══════════════════════════════
-- SORTED SET: Game leaderboard
-- ═══════════════════════════════

-- Add players with their scores
-- Score is the second argument, member is the third
ZADD game:leaderboard 4850 "player:alice"
ZADD game:leaderboard 6200 "player:bob"
ZADD game:leaderboard 5930 "player:carol"
ZADD game:leaderboard 7100 "player:dave"
ZADD game:leaderboard 3200 "player:eve"

-- Update alice's score after she beats a level (just ZADD again)
ZADD game:leaderboard 5500 "player:alice"

-- Top 3 players, highest score first (WITHSCORES shows the score too)
ZREVRANGE game:leaderboard 0 2 WITHSCORES

-- What rank is carol? (0-indexed, ZREVRANK = rank by descending score)
ZREVRANK game:leaderboard "player:carol"
-- Returns: (integer) 2  (3rd place, 0-indexed)

-- All players with scores between 5000 and 7000
ZRANGEBYSCORE game:leaderboard 5000 7000 WITHSCORES

-- How many players scored over 5000?
ZCOUNT game:leaderboard 5001 +inf
-- Returns: (integer) 3

Bitmaps and HyperLogLog — Efficient Counting and Boolean Operations

Bitmaps are not a distinct type — they're just Strings on which you can perform bit-level operations. Use SETBIT and GETBIT to manipulate individual bits. That gives you an incredibly memory-efficient way to store boolean flags: one user's daily sign-in for a year takes just 365 bits — about 46 bytes. Compare that to storing a String per day.

Bit operations like BITCOUNT, BITOP (AND, OR, XOR, NOT) let you compute retention, cohort analysis, and feature flags across millions of users in a single command. For example, BITOP AND daily:2026-03-01 daily:2026-03-02 shows users active on both days.

HyperLogLog is a probabilistic data structure for approximate cardinality counting with fixed memory. Each key takes ~12KB regardless of how many unique items you add. PFADD adds elements, PFCOUNT returns the approximate count (typical error 0.81%). It's perfect for counting unique visitors, search queries, or IPs where precision within a few percent is acceptable.

Use cases: daily active users (HyperLogLog), feature flags (Bitmap), user behaviour tracking (Bitmap).

-- ═══════════════════════════════
-- BITMAP: Track daily sign-ins
-- ═══════════════════════════════

-- User 42 signs in on day 0 of the year (Jan 1)
SETBIT signin:2026 42 1
-- (integer) 0  (was 0 before)

-- User 42 signs in again on day 100 (Apr 10)
SETBIT signin:2026 100 1

-- Has user 42 signed in at all?
GETBIT signin:2026 42
-- Returns: (integer) 1

-- How many users signed in on day 100?
BITCOUNT signin:2026 0 199

-- Users active on both day 100 and day 101? (intersection)
BITOP AND active_both signin:2026 day100 day101

-- ═══════════════════════════════
-- HYPERLOGLOG: Unique visitors
-- ═══════════════════════════════

-- Add visitor IPs to the hyperloglog key for today
PFADD visitors:2026-03-25 "192.168.1.1" "10.0.0.2" "172.16.0.3"
PFADD visitors:2026-03-25 "192.168.1.1"  -- duplicate, ignored

-- Approximate unique count
PFCOUNT visitors:2026-03-25
-- Returns: (integer) 3

-- Merge yesterday and today for weekly unique visitors
PFMERGE visitors:week12 visitors:2026-03-24 visitors:2026-03-25
PFCOUNT visitors:week12

Analytical Structures — Bitmaps and HyperLogLog

While Strings, Hashes, Lists, and Sets handle transactional data, Bitmaps and HyperLogLog are purpose-built for analytical queries at scale. They trade off some features (range queries, field updates) for extreme memory efficiency and server-side aggregation.

Bitmaps let you store boolean arrays compactly. With BITCOUNT and BITOP you can answer questions like 'how many users logged in on both Monday and Tuesday?' across millions of users in microseconds — no scanning, no joins. The memory cost is 1 bit per user per day; a year of daily tracking for 10 million users costs ~456 MB.

HyperLogLog solves the 'how many unique items?' problem with a fixed memory of ~12KB per key, regardless of how many millions of distinct elements you add. The error is typically 0.81%, which is acceptable for dashboards and trend analysis. PFMERGE lets you combine multiple days into a weekly or monthly count without storing individual elements.

Choose Bitmaps when your user base is a known integer range (e.g., user IDs from 1 to 10 million) and you need exact per-day boolean status. Choose HyperLogLog when you need unique counts of arbitrary strings (IPs, search queries, email addresses) and can tolerate small errors.

Production tip: Bitmaps are not an independent type — they are Strings. If you need to shard across multiple keys (for >2^32 bits), use consistent hashing on the user ID to select the key.

-- Analytical example: 7-day active user count
-- Day 1: users with ID 1,2,3,4,5
SETBIT active:day1 1 1
SETBIT active:day1 2 1
SETBIT active:day1 3 1
SETBIT active:day1 4 1
SETBIT active:day1 5 1

-- Day 2: users 3,4,5,6,7
SETBIT active:day2 3 1
SETBIT active:day2 4 1
SETBIT active:day2 5 1
SETBIT active:day2 6 1
SETBIT active:day2 7 1

-- Users active on both days (intersection)
BITOP AND active_both active:day1 active:day2
BITCOUNT active_both
-- Returns: 3 (users 3,4,5)

-- Total unique users across both days using HyperLogLog
PFADD unique:day1 user1 user2 user3 user4 user5
PFADD unique:day2 user3 user4 user5 user6 user7
PFCOUNT unique:day1 unique:day2
-- Returns: 7 (all unique users)

Redis Streams — The Append-Only Log for Event Sourcing

Redis Streams (introduced in Redis 5.0) is an append-only log data structure. Each stream entry has a unique auto-generated ID (timestamp-sequence) and a set of field-value pairs — essentially a persistent, ordered, and replayable event log.

The key features are: non-destructive reads (consumers can read the same entry multiple times), consumer groups (competing consumers with acknowledgments), and blocking reads (like BRPOP but for streams). This makes Streams the natural choice for event sourcing, message queues with persistence, and audit trails.

Unlike Lists, entries in a stream are never removed by reading — they stay until explicitly trimmed (XTRIM) or evicted by maxlen. This allows replay and multiple consumer groups. Use XADD to append, XREAD to read in blocking mode, XREADGROUP for consumer groups, and XACK to acknowledge processed messages.

Streams also support range queries by ID, so you can replay historical events from a given point — invaluable for debugging or rebuilding state.

-- ═══════════════════════════════
-- STREAM: Order event log
-- ═══════════════════════════════

-- Add an event to the order stream (auto-generated ID using *)
XADD order:events * event_name "order_placed" user_id "42" amount "29.99"
-- Returns: "1700000000000-0" (timestamp-sequence)

XADD order:events * event_name "payment_received" user_id "42" amount "29.99"

XADD order:events * event_name "shipped" user_id "42" tracking "1Z999AA10123456784"

-- Read all events from the start (blocking, latest offset $)
-- Non-blocking: read from start to latest
XREAD COUNT 100 STREAMS order:events 0

-- Block until new events arrive (like BRPOP)
XREAD BLOCK 5000 STREAMS order:events $

-- Create a consumer group for competing consumers
XGROUP CREATE order:events order-processors $ MKSTREAM

-- Consumer 1: read and claim
XREADGROUP GROUP order-processors consumer-1 COUNT 10 BLOCK 2000 STREAMS order:events >

-- Acknowledge processing
XACK order:events order-processors 1700000000000-0

-- Trim the stream to keep only the last 1000 entries
XTRIM order:events MAXLEN ~ 1000
-- ~ means approximate trimming for efficiency

Memory Overcommit — Why Your Sorted Set Will Eat RAM for Breakfast

You picked Redis because it's fast. Speed costs memory. Every data structure has a memory footprint that scales non-linearly. Sorted Sets look innocent — score, member, done. But internally Redis uses a skip list plus a hash table. For 10 million members, that's roughly 1.2 GB of overhead before you store a single byte of actual data. Strings are worse when used as counters — each key carries ~90 bytes of metadata. If you store 50 million session tokens as individual keys, you've burned 4.5 GB on structure alone. The competing pages never show you the bill. They show you the pretty syntax. Here's the fix: benchmark with redis-cli --bigkeys before you commit to production. Watch the memory/expiry ratio. Set a maxmemory policy that matches your data pattern — not the default noeviction. Your Sorted Set leaderboard should be backed by a capped ZREMRANGEBYRANK maintenance job. Run it every N writes. Not on a cron. On write. Production doesn't wait for cron.

// io.thecodeforge — database tutorial

// Audit memory per key type — run before any schema decision
redis-cli --bigkeys

// Example output for a production box with 12 GB dataset

// Biggest keys by type
// Strings: 2.3M keys, avg size 84 bytes, total 193 MB
// Hashes: 140K keys, avg size 2.1 KB, total 287 MB
// Sorted Sets: 850 keys, avg size 1.4 MB, total 1.19 GB
// Lists: 12 keys, avg size 45 MB, total 540 MB

// Watch the overhead: skip list + hash table = 2x raw data
redis-cli MEMORY USAGE leaderboard:q4_2025
// (integer) 2234832

Persistence Semantics — RDB vs AOF vs No Save. Pick One or Lose Data

Competitors treat persistence as an afterthought. 'Turn on RDB and you're safe.' That's a lie. RDB snapshots are point-in-time. Lose the last 60 seconds of writes and your leaderboard resets. AOF logs every write — but at fsync every second, you still lose one second per crash. The only zero-loss path is AOF with appendfsync always, which kills throughput by 10x. Nobody tells you that. Here's what they miss: your data structure choice determines your acceptable loss. Streams with consumer groups demand AOF. Cached session tokens? No persistence needed — rebuild on restart. Sorted Set leaderboards for a game? RDB every 5 minutes is fine because you can replay from game events. The real sin is mixing persistence tolerances in one Redis instance. Your analytics pipeline runs HyperLogLog with no persistence. Your payment queue runs Streams with AOF. Same box, different guarantees. Split them. Run one Redis for ephemeral, one for durable. Senior engineers ship two configs, not one.

// io.thecodeforge — database tutorial

// Ephemeral cache instance — no persistence, max throughput
save ""
appendonly no
maxmemory-policy allkeys-lru

// Durable queue instance — AOF every write, no compromise
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync always
aof-use-rdb-preamble yes

// Check actual AOF rewrite lag in production
redis-cli INFO Persistence
// aof_rewrite_in_progress:0
// aof_last_rewrite_time_sec:12

Command Pipelining — The Single Biggest Performance Lever You're Ignoring

Every competitor shows you how to SET a key. None show you the cost of round-trips. At 100 microseconds per command on a local network, 10,000 pipelined commands take 10 seconds — one at a time. Pipeline them and it's 500 milliseconds. Twenty times faster. That's not theory. That's a production incident where your batch job for Sorted Set leaderboard updates locked the app for 12 seconds because you sent ZADD one by one. The fix: batch into chunks of 500-1000 commands. Use pipelining for all bulk writes — Hash set, List push, Stream XADD. But watch the pipeline buffer. Send too many and you'll OOM the client. Redis 7.4 improved this with client-side caching, but 90% of you are on older versions. For Streams, use XADD with MAXLEN ~1000 in batch — don't trim per entry. For Sets, SADD 500 members at once. The competitor tutorials treat Redis as a single-command store. It's not. It's a pipeline engine. Treat it like one.

// io.thecodeforge — database tutorial

// Batch ZADD for leaderboard updates — 750 members at once
local pipeline = redis.pipeline()
for i = 1, 750 do
  pipeline:zadd("leaderboard:arena:2025-04", {score = math.random(1000, 999999), member = "player_" .. i})
end
local replies = pipeline:execute()

// VS naive loop — 750 round trips, ~7 seconds
for i = 1, 750 do
  redis.call("ZADD", "leaderboard:arena:2025-04", math.random(1000, 999999), "player_" .. i)
end

// Measure the difference
redis-cli --latency -h localhost -p 6379
// min: 0, max: 3, avg: 0.12 (milliseconds) — pipeline wins hard