Design TinyURL — Cache Stampede & Viral Link Failures

Every senior engineer has sat across from an interviewer who says 'design a URL shortener' with a calm smile. It sounds trivial — take a long URL, make it short. But behind that smile is a question that probes distributed systems, database design, caching strategy, hash collision handling, rate limiting, analytics, and horizontal scaling simultaneously. Bit.ly processes over 600 million redirects per day. TinyURL has been alive since 2002. These systems are deceptively simple on the surface and genuinely hard to build correctly at scale.

The core problem is a deceptively asymmetric one: writes are rare, reads are overwhelmingly frequent. When you shorten a URL, that's a one-time write. But that short link might be embedded in a viral tweet and hit 10 million times in an hour. Your design has to reflect this read-heavy reality — every architectural choice from your hashing scheme to your cache eviction policy flows from that single insight.

By the end of this article you'll be able to walk into any system design interview and design TinyURL end-to-end: justify your short code generation strategy, design a DB schema that survives traffic spikes, build a caching layer that handles 99% of reads from memory, handle custom aliases and expiration, discuss analytics pipelines, and correctly answer every follow-up an interviewer throws at you. Let's build it.

Why TinyURL Design Tests More Than URL Shortening

The TinyURL design interview asks you to architect a URL shortening service — a system that maps long URLs to short, unique aliases and redirects clients on access. The core mechanic is a key-value lookup: given a short key (e.g., 7 characters from base62), return the original URL and issue an HTTP 302 redirect. This problem is a systems design classic because it forces you to reason about read-heavy workloads, collision-free key generation, and caching under extreme traffic.

In practice, the service must handle billions of writes (new URLs) and tens of billions of reads (redirects). Key properties that matter: key generation must be idempotent and collision-resistant (using distributed counters or pre-generated keys), redirect latency must stay under 10ms at P99, and the system must survive traffic spikes from viral links. A naive cache with a single Redis instance will collapse under a cache stampede when a popular link goes viral — every miss triggers a database read, overwhelming the DB and causing cascading failures.

You use this design pattern when you need a globally unique, short identifier for a resource and expect asymmetric read/write ratios (100:1 or higher). It matters in real systems because the same principles apply to CDN edge caching, distributed ID generation (Snowflake), and rate-limited API gateways. Getting the cache invalidation and key distribution wrong is the #1 cause of production outages in URL shorteners.

The Core Logic: Base62 Encoding vs. Hashing

In a URL shortener, the 'Magic' is how we generate the tiny string. You have two main paths: Hashing (MD5/SHA-256) or Base62 Encoding a unique ID. Hashing often leads to collisions that require complex 'check-and-retry' logic. The industry-standard approach is to use a distributed ID generator (like a Snowflake ID or a centralized Range Manager) and convert that numeric ID into a Base62 string (a-z, A-Z, 0-9).

For example, an ID like 125 converted to Base62 results in a short, predictable, and unique string. To prevent predictability (so people can't guess the 'next' URL), we can add a bit of salt or shuffle our Base62 alphabet.

package io.thecodeforge.shortener;

/**
 * TheCodeForge Production-Grade Base62 Encoder
 * Converts a unique Long ID into a 7-character short code.
 */
public class Base62Encoder {
    private static final String ALPHABET = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
    private static final int BASE = ALPHABET.length();

    public static String encode(long id) {
        StringBuilder sb = new StringBuilder();
        while (id > 0) {
            sb.append(ALPHABET.charAt((int) (id % BASE)));
            id /= BASE;
        }
        // Pad to ensure consistent length if required by business logic
        while (sb.length() < 7) {
            sb.append(ALPHABET.charAt(0));
        }
        return sb.reverse().toString();
    }

    public static void main(String[] args) {
        long uniqueId = 56800235584L; // Example ID from a distributed generator
        System.out.println("Short Code for " + uniqueId + ": " + encode(uniqueId));
    }
}

Data Layer Strategy: Handling Scale and Redirection

Since this is a read-heavy system (100:1 read/write ratio), our database choice and caching strategy are critical. We use a NoSQL database like Cassandra or a sharded MongoDB for the URL mappings because we don't need complex joins—just a simple Key-Value lookup.

To achieve sub-millisecond redirects, we put a Redis cache in front of the database. We use an LRU (Least Recently Used) eviction policy because in the real world, 20% of the links (the viral ones) will generate 80% of the traffic.

-- io.thecodeforge.shortener - Database Schema
-- Optimized for NoSQL or Sharded SQL

CREATE TABLE io_thecodeforge.url_mapping (
    short_key    VARCHAR(7) PRIMARY KEY, 
    original_url TEXT NOT NULL,
    user_id      BIGINT,
    created_at   TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at   TIMESTAMP,
    click_count  BIGINT DEFAULT 0
);

-- Secondary Index for User Management
CREATE INDEX idx_user_urls ON io_thecodeforge.url_mapping(user_id);

Distributed ID Generation: The Backbone of Uniqueness

The unique ID that feeds into Base62 encoding must be globally unique across all servers. A simple auto-increment DB column doesn't scale — you'd have a single point of contention. The standard pattern is to use a distributed ID generator. Two common approaches: Snowflake (Twitter's algorithm) and ZooKeeper-managed ID ranges.

Snowflake generates 64-bit IDs: timestamp (41 bits) + machine ID (10 bits) + sequence (12 bits). This gives 4096 IDs per millisecond per machine, and the IDs are time-sortable. ZooKeeper assigns a range of IDs (e.g., 0-100000) to each app server; when exhausted, the server requests a new range. Both avoid collisions without a central DB write bottleneck.

In production, you'll also want to make the short code appear random. You can shuffle the Base62 alphabet permanently or XOR the ID with a secret before encoding. That prevents users from guessing sequential short codes and scraping all URLs.

package io.thecodeforge.shortener;

/**
 * Simplified Snowflake ID generator for TheCodeForge URL shortener.
 * Uses: 41 bits for timestamp (ms), 10 bits for machine ID, 12 bits for sequence.
 */
public class SnowflakeIdGenerator {
    private final long machineId;
    private long lastTimestamp = -1L;
    private long sequence = 0L;

    public SnowflakeIdGenerator(long machineId) {
        if (machineId > 1023) throw new IllegalArgumentException("Machine ID must be <= 1023");
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = System.currentTimeMillis();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("Clock moved backwards!");
        }
        if (timestamp == lastTimestamp) {
            sequence = (sequence + 1) & 4095; // 12-bit mask
            if (sequence == 0) {
                // Wait for next millisecond
                while ((timestamp = System.currentTimeMillis()) <= lastTimestamp) { }
            }
        } else {
            sequence = 0;
        }
        lastTimestamp = timestamp;
        return (timestamp - 1704067200000L) << 22 | (machineId << 12) | sequence;
    }

    public static void main(String[] args) {
        SnowflakeIdGenerator gen = new SnowflakeIdGenerator(1);
        System.out.println("Generated ID: " + gen.nextId());
    }
}

Caching Strategy: Surviving the Viral Spike

We already mentioned a Redis cluster with LRU eviction. But to really survive a viral spike, you need a multi-layer caching strategy. The first layer is an in-memory cache (like Caffeine or Guava on each application server) that holds the hottest entries with a very short TTL (1-2 seconds). The second layer is a Redis cluster, and the third is the database.

When a request arrives, the app server checks its local cache first. On miss, it queries Redis. On Redis miss, it queries the database and then populates both caches. To prevent a stampede (thundering herd) when a cached key expires, use a distributed lock or a get-or-compute pattern. Only one thread should reload a cache entry; others should wait or serve a stale value.

For short codes that go viral, you can proactively pin them to dedicated cache nodes or increase their priority. Use consistent hashing for the Redis cluster so that adding nodes doesn't cause mass cache invalidation.

package io.thecodeforge.shortener;

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.sync.RedisCommands;

import java.util.concurrent.TimeUnit;

public class CacheService {
    private final Cache<String, String> localCache;
    private final RedisCommands<String, String> redis;

    public CacheService(RedisClient redisClient) {
        this.localCache = Caffeine.newBuilder()
                .maximumSize(10_000)
                .expireAfterWrite(2, TimeUnit.SECONDS)
                .build();
        this.redis = redisClient.connect().sync();
    }

    public String getOriginalUrl(String shortKey) {
        // Check local cache first (fastest)
        String url = localCache.getIfPresent(shortKey);
        if (url != null) return url;

        // Check Redis
        url = redis.get(shortKey);
        if (url != null) {
            localCache.put(shortKey, url);
            return url;
        }

        // Miss all caches — fetch from DB and populate
        url = fetchFromDatabase(shortKey);
        if (url != null) {
            redis.setex(shortKey, 3600, url); // 1 hour TTL
            localCache.put(shortKey, url);
        }
        return url;
    }

    private String fetchFromDatabase(String shortKey) {
        // Implementation: query Cassandra or sharded MySQL
        return null;
    }
}

Analytics and Click Tracking Pipeline

A URL shortener is not just about redirection — it's a data business. Every click is valuable analytical data: geo-location, referrer, user agent, timestamp. You can't afford to write this data synchronously during a redirect (that would add latency). The pattern is asynchronous: the web server publishes a click event to a message queue (Kafka) and returns the 302/301 immediately. A separate consumer processes these events and updates the click_count in the database and aggregates data for dashboards.

Kafka topics can be partitioned by short key to maintain ordering per URL. The consumer can batch updates to the database (e.g., update click_count = click_count + 1 for 100 events at once). For real-time analytics, use a stream processor (Spark Streaming, Flink) to compute counters down to 1-minute granularity.

We also need to handle deduplication: users may refresh or multiple bots may click. Use a combination of IP + user agent + timestamp window to filter duplicates, or accept a small error percentage (most shorteners tolerate 1-2% overcount).

package io.thecodeforge.shortener;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class ClickEventPublisher {
    private static final Logger log = LoggerFactory.getLogger(ClickEventPublisher.class);
    private final KafkaProducer<String, String> producer;
    private final String topic = "url_clicks";

    public ClickEventPublisher(Properties kafkaProps) {
        this.producer = new KafkaProducer<>(kafkaProps);
    }

    public void publishClick(String shortKey, String userAgent, String ip, long timestamp) {
        String value = shortKey + "|" + userAgent + "|" + ip + "|" + timestamp;
        producer.send(new ProducerRecord<>(topic, shortKey, value), (meta, ex) -> {
            if (ex != null) log.error("Failed to publish click for " + shortKey, ex);
        });
    }

    public void close() {
        producer.close();
    }
}

Why Your First Base62 Implementation Will Burn in Production

Every junior engineer starts with the same trap: integer ID → Base62 string → short URL. Simple. Elegant. Wrong for any system that survives more than a single server reboot.

The problem? The conversion is reversible. Anyone who gets a short URL can enumerate your entire ID space. They can scrape every URL you've ever shortened. Your competitor can map your traffic patterns. Your private links become public.

Production systems don't use sequential IDs for exactly this reason. You need unpredictable short codes. The industry standard is a random token (system-generated UUID or Snowflake ID) that has zero correlation to the storage key. Base62 only enters the picture when you need a human-readable representation of that random token.

But wait — random tokens collide. That's fine. You detect the collision, regenerate, and retry. At 62^6 possibilities, collisions are statistically irrelevant at any sane scale. The real cost is the retry overhead in your write path.

// io.thecodeforge — interview tutorial

import uuid
import base62
from typing import Optional

class ShortCodeGenerator:
    def __init__(self, max_retries: int = 3):
        self.max_retries = max_retries
        self._seen_codes = set()

    def generate(self) -> str:
        for attempt in range(self.max_retries):
            token = uuid.uuid4().int & ((1 << 36) - 1)  # 36-bit random
            code = base62.encode(token)[:6]
            if code not in self._seen_codes:
                self._seen_codes.add(code)
                return code
        raise RuntimeError(f"Collision after {self.max_retries} retries — improbable at scale")

gen = ShortCodeGenerator()
for _ in range(5):
    print(f"Short code: {gen.generate()}")

How TikTok Handles the Viral Spike That Kills Naive Caches

Your caching strategy looks great on paper. 80% cache hit rate. Redis cluster with replication. Eviction policy set to LRU. Then a celebrity tweets your shortened link and your cache gets eviscerated.

The problem isn't the hot key — it's the thundering herd of cold keys. A viral event means millions of requests for URLs that have never been cached. Every one of those requests hits your database. The database melts. The site goes dark.

The fix is shockingly simple: cache-aside with a distributed mutex. Before hitting the database for a cache miss, acquire a lightweight lock (Redis SETNX) scoped to the short code. Only the first requestor actually queries the database. The rest wait a few milliseconds and retry the cache.

TikTok's approach goes further: they pre-warm the cache for known high-traffic content. For TinyURL, that means tracking URL creation velocity. If a new short URL gets 100 redirects in its first minute, it's categorized as "viral candidate" and all its metadata gets promoted to the L1 cache tier proactively.

// io.thecodeforge — interview tutorial

import redis
import time
from typing import Optional

cache = redis.Redis(connection_pool=redis.ConnectionPool(max_connections=100))
MUTEX_TTL = 5  # seconds
CACHE_TTL = 3600

def resolve_short_url(short_code: str) -> Optional[str]:
    long_url = cache.get(f"short:{short_code}")
    if long_url:
        return long_url

    # Distributed mutex — only one process hits the DB
    lock_key = f"lock:{short_code}"
    if cache.setnx(lock_key, "1"):
        cache.expire(lock_key, MUTEX_TTL)
        long_url = query_database(short_code)  # real DB call
        if long_url:
            cache.setex(f"short:{short_code}", CACHE_TTL, long_url)
        cache.delete(lock_key)
        return long_url

    # Wait and retry — up to 50ms typical
    time.sleep(0.01)
    return resolve_short_url(short_code)

The Database Sharding Strategy Nobody Teaches You

Every blog post tells you to shard by user ID. Great for Instagram. Terrible for TinyURL. A single user creating 10,000 URLs per second is a normal day. Sharding by user means one hot shard handles all writes for power users while others sit idle.

The better approach: shard by the short code's first character. With 62 possible first characters, you get automatic load distribution. The write throughput is uniform because short codes are random. Read throughput follows the same pattern — viral URLs spread evenly across shards.

But here's the gotcha: range queries on creation time become impossible. Need to find all URLs created in the last hour? You must query all shards. That's fine for analytics — you batch those queries and accept the latency. The redirect path stays fast because it's a point lookup.

Pro tip: use consistent hashing with virtual nodes on the short code. If you add a shard, only 1/62nd of your data moves. You don't need to rebalance the entire cluster.

// io.thecodeforge — interview tutorial

import hashlib
from typing import List, Dict

class ShardRouter:
    def __init__(self, shard_endpoints: List[str]):
        self.virtual_nodes: Dict[int, str] = {}
        for shard in shard_endpoints:
            for vnode in range(128):  # 128 virtual nodes per shard
                key = hashlib.md5(f"{shard}:{vnode}".encode()).hexdigest()
                self.virtual_nodes[int(key[:8], 16)] = shard

    def get_shard(self, short_code: str) -> str:
        # Hash the short code to find its virtual node
        hash_val = int(hashlib.md5(short_code.encode()).hexdigest()[:8], 16)
        sorted_keys = sorted(self.virtual_nodes.keys())
        for key in sorted_keys:
            if hash_val <= key:
                return self.virtual_nodes[key]
        return self.virtual_nodes[sorted_keys[0]]  # wrap around

router = ShardRouter(["shard-db-01", "shard-db-02", "shard-db-03"])
print(f"aB3xYz -> {router.get_shard('aB3xYz')}")
print(f"9kLmNp -> {router.get_shard('9kLmNp')}")

Alternative Approaches: Snowflake vs Redis vs Database Sequences

Choosing the right ID generation strategy is critical for TinyURL's scalability. Three common approaches are Snowflake, Redis, and database sequences. Snowflake (used by Twitter) generates 64-bit unique IDs using a timestamp, worker ID, and sequence number. It's decentralized and fast but requires clock synchronization. Redis offers atomic INCR commands with optional persistence, providing low-latency ID generation but introducing a single point of failure unless clustered. Database sequences (e.g., PostgreSQL SERIAL) are simple but become a bottleneck under high write loads. For TinyURL, Snowflake is ideal for distributed systems needing high throughput, while Redis suits moderate scales with caching needs. Database sequences are best for small deployments. Example: Snowflake ID = timestamp (41 bits) + worker ID (10 bits) + sequence (12 bits).

import time
import threading

class Snowflake:
    def __init__(self, worker_id, datacenter_id):
        self.worker_id = worker_id
        self.datacenter_id = datacenter_id
        self.sequence = 0
        self.last_timestamp = -1
        self.lock = threading.Lock()

    def _timestamp(self):
        return int(time.time() * 1000)

    def generate(self):
        with self.lock:
            timestamp = self._timestamp()
            if timestamp < self.last_timestamp:
                raise Exception("Clock moved backwards")
            if timestamp == self.last_timestamp:
                self.sequence = (self.sequence + 1) & 4095
                if self.sequence == 0:
                    timestamp = self._wait_next_millis()
            else:
                self.sequence = 0
            self.last_timestamp = timestamp
            return (timestamp << 22) | (self.datacenter_id << 17) | (self.worker_id << 12) | self.sequence

    def _wait_next_millis(self):
        timestamp = self._timestamp()
        while timestamp <= self.last_timestamp:
            timestamp = self._timestamp()
        return timestamp

TinyURL with Analytics: Click Tracking and Dashboards

Analytics are essential for understanding link performance. Click tracking involves capturing each redirect event with metadata: timestamp, IP address, user agent, referrer, and geolocation. This data is streamed to a message queue (e.g., Kafka) and processed asynchronously to avoid slowing down redirects. A separate analytics service aggregates data into time-series databases (e.g., InfluxDB) or OLAP stores (e.g., ClickHouse) for dashboard queries. Dashboards display metrics like total clicks, unique visitors, geographic distribution, and click-through rates over time. For real-time updates, use WebSocket connections or periodic polling. Example: A TinyURL click event triggers a POST to /analytics with payload {short_code, timestamp, ip, user_agent}. The analytics service enriches the IP with GeoIP data and writes to Kafka. A consumer updates Redis sorted sets for hourly counts and ClickHouse for long-term storage.

from flask import Flask, request, jsonify
import kafka_producer

app = Flask(__name__)
producer = kafka_producer.KafkaProducer(bootstrap_servers='localhost:9092')

@app.route('/redirect/<short_code>')
def redirect(short_code):
    # ... resolve URL logic ...
    # Track click asynchronously
    click_data = {
        'short_code': short_code,
        'timestamp': int(time.time()),
        'ip': request.remote_addr,
        'user_agent': request.headers.get('User-Agent'),
        'referrer': request.headers.get('Referer')
    }
    producer.send('clicks', value=click_data)
    return redirect(target_url, 302)

Custom Short URLs: Base62 vs Base64URL Encoding Comparison

Custom short URLs often use encoding schemes to represent numeric IDs as short strings. Base62 uses 62 characters (a-z, A-Z, 0-9) and is case-sensitive, producing strings like 'abc123'. Base64URL is a variant of Base64 that replaces '+' and '/' with '-' and '_' to be URL-safe, using 64 characters. Base62 is more human-readable and avoids ambiguous characters (e.g., 'l' vs '1'), but Base64URL is more compact (shorter strings for the same numeric range). For example, encoding ID 123456789: Base62 yields '8M0kX' (5 chars), while Base64URL yields '7cDf' (4 chars). However, Base64URL may include '-' and '_' which can be less user-friendly. For TinyURL, Base62 is preferred for custom short URLs because it's easier to type and remember. Base64URL is better for machine-generated links where compactness matters. Both require padding removal and careful handling of collisions.

import base64
import string

BASE62_ALPHABET = string.ascii_letters + string.digits

def base62_encode(num):
    if num == 0:
        return BASE62_ALPHABET[0]
    result = []
    while num > 0:
        num, rem = divmod(num, 62)
        result.append(BASE62_ALPHABET[rem])
    return ''.join(reversed(result))

def base64url_encode(num):
    # Encode integer to bytes then base64url
    num_bytes = num.to_bytes((num.bit_length() + 7) // 8, 'big')
    return base64.urlsafe_b64encode(num_bytes).rstrip(b'=').decode()

# Example
print(base62_encode(123456789))  # 8M0kX
print(base64url_encode(123456789))  # B7cDf