Design TinyURL — Cache Stampede & Viral Link Failures

Every senior engineer has sat across from an interviewer who says 'design a URL shortener' with a calm smile. It sounds trivial — take a long URL, make it short. But behind that smile is a question that probes distributed systems, database design, caching strategy, hash collision handling, rate limiting, analytics, and horizontal scaling simultaneously. Bit.ly processes over 600 million redirects per day. TinyURL has been alive since 2002. These systems are deceptively simple on the surface and genuinely hard to build correctly at scale.

The core problem is a deceptively asymmetric one: writes are rare, reads are overwhelmingly frequent. When you shorten a URL, that's a one-time write. But that short link might be embedded in a viral tweet and hit 10 million times in an hour. Your design has to reflect this read-heavy reality — every architectural choice from your hashing scheme to your cache eviction policy flows from that single insight.

By the end of this article you'll be able to walk into any system design interview and design TinyURL end-to-end: justify your short code generation strategy, design a DB schema that survives traffic spikes, build a caching layer that handles 99% of reads from memory, handle custom aliases and expiration, discuss analytics pipelines, and correctly answer every follow-up an interviewer throws at you. Let's build it.

The Core Logic: Base62 Encoding vs. Hashing

In a URL shortener, the 'Magic' is how we generate the tiny string. You have two main paths: Hashing (MD5/SHA-256) or Base62 Encoding a unique ID. Hashing often leads to collisions that require complex 'check-and-retry' logic. The industry-standard approach is to use a distributed ID generator (like a Snowflake ID or a centralized Range Manager) and convert that numeric ID into a Base62 string (a-z, A-Z, 0-9).

For example, an ID like 125 converted to Base62 results in a short, predictable, and unique string. To prevent predictability (so people can't guess the 'next' URL), we can add a bit of salt or shuffle our Base62 alphabet.

package io.thecodeforge.shortener;

/**
 * TheCodeForge Production-Grade Base62 Encoder
 * Converts a unique Long ID into a 7-character short code.
 */
public class Base62Encoder {
    private static final String ALPHABET = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
    private static final int BASE = ALPHABET.length();

    public static String encode(long id) {
        StringBuilder sb = new StringBuilder();
        while (id > 0) {
            sb.append(ALPHABET.charAt((int) (id % BASE)));
            id /= BASE;
        }
        // Pad to ensure consistent length if required by business logic
        while (sb.length() < 7) {
            sb.append(ALPHABET.charAt(0));
        }
        return sb.reverse().toString();
    }

    public static void main(String[] args) {
        long uniqueId = 56800235584L; // Example ID from a distributed generator
        System.out.println("Short Code for " + uniqueId + ": " + encode(uniqueId));
    }
}

Data Layer Strategy: Handling Scale and Redirection

Since this is a read-heavy system (100:1 read/write ratio), our database choice and caching strategy are critical. We use a NoSQL database like Cassandra or a sharded MongoDB for the URL mappings because we don't need complex joins—just a simple Key-Value lookup.

To achieve sub-millisecond redirects, we put a Redis cache in front of the database. We use an LRU (Least Recently Used) eviction policy because in the real world, 20% of the links (the viral ones) will generate 80% of the traffic.

-- io.thecodeforge.shortener - Database Schema
-- Optimized for NoSQL or Sharded SQL

CREATE TABLE io_thecodeforge.url_mapping (
    short_key    VARCHAR(7) PRIMARY KEY, 
    original_url TEXT NOT NULL,
    user_id      BIGINT,
    created_at   TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at   TIMESTAMP,
    click_count  BIGINT DEFAULT 0
);

-- Secondary Index for User Management
CREATE INDEX idx_user_urls ON io_thecodeforge.url_mapping(user_id);

Distributed ID Generation: The Backbone of Uniqueness

The unique ID that feeds into Base62 encoding must be globally unique across all servers. A simple auto-increment DB column doesn't scale — you'd have a single point of contention. The standard pattern is to use a distributed ID generator. Two common approaches: Snowflake (Twitter's algorithm) and ZooKeeper-managed ID ranges.

Snowflake generates 64-bit IDs: timestamp (41 bits) + machine ID (10 bits) + sequence (12 bits). This gives 4096 IDs per millisecond per machine, and the IDs are time-sortable. ZooKeeper assigns a range of IDs (e.g., 0-100000) to each app server; when exhausted, the server requests a new range. Both avoid collisions without a central DB write bottleneck.

In production, you'll also want to make the short code appear random. You can shuffle the Base62 alphabet permanently or XOR the ID with a secret before encoding. That prevents users from guessing sequential short codes and scraping all URLs.

package io.thecodeforge.shortener;

/**
 * Simplified Snowflake ID generator for TheCodeForge URL shortener.
 * Uses: 41 bits for timestamp (ms), 10 bits for machine ID, 12 bits for sequence.
 */
public class SnowflakeIdGenerator {
    private final long machineId;
    private long lastTimestamp = -1L;
    private long sequence = 0L;

    public SnowflakeIdGenerator(long machineId) {
        if (machineId > 1023) throw new IllegalArgumentException("Machine ID must be <= 1023");
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = System.currentTimeMillis();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("Clock moved backwards!");
        }
        if (timestamp == lastTimestamp) {
            sequence = (sequence + 1) & 4095; // 12-bit mask
            if (sequence == 0) {
                // Wait for next millisecond
                while ((timestamp = System.currentTimeMillis()) <= lastTimestamp) { }
            }
        } else {
            sequence = 0;
        }
        lastTimestamp = timestamp;
        return (timestamp - 1704067200000L) << 22 | (machineId << 12) | sequence;
    }

    public static void main(String[] args) {
        SnowflakeIdGenerator gen = new SnowflakeIdGenerator(1);
        System.out.println("Generated ID: " + gen.nextId());
    }
}

Caching Strategy: Surviving the Viral Spike

We already mentioned a Redis cluster with LRU eviction. But to really survive a viral spike, you need a multi-layer caching strategy. The first layer is an in-memory cache (like Caffeine or Guava on each application server) that holds the hottest entries with a very short TTL (1-2 seconds). The second layer is a Redis cluster, and the third is the database.

When a request arrives, the app server checks its local cache first. On miss, it queries Redis. On Redis miss, it queries the database and then populates both caches. To prevent a stampede (thundering herd) when a cached key expires, use a distributed lock or a get-or-compute pattern. Only one thread should reload a cache entry; others should wait or serve a stale value.

For short codes that go viral, you can proactively pin them to dedicated cache nodes or increase their priority. Use consistent hashing for the Redis cluster so that adding nodes doesn't cause mass cache invalidation.

package io.thecodeforge.shortener;

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.sync.RedisCommands;

import java.util.concurrent.TimeUnit;

public class CacheService {
    private final Cache<String, String> localCache;
    private final RedisCommands<String, String> redis;

    public CacheService(RedisClient redisClient) {
        this.localCache = Caffeine.newBuilder()
                .maximumSize(10_000)
                .expireAfterWrite(2, TimeUnit.SECONDS)
                .build();
        this.redis = redisClient.connect().sync();
    }

    public String getOriginalUrl(String shortKey) {
        // Check local cache first (fastest)
        String url = localCache.getIfPresent(shortKey);
        if (url != null) return url;

        // Check Redis
        url = redis.get(shortKey);
        if (url != null) {
            localCache.put(shortKey, url);
            return url;
        }

        // Miss all caches — fetch from DB and populate
        url = fetchFromDatabase(shortKey);
        if (url != null) {
            redis.setex(shortKey, 3600, url); // 1 hour TTL
            localCache.put(shortKey, url);
        }
        return url;
    }

    private String fetchFromDatabase(String shortKey) {
        // Implementation: query Cassandra or sharded MySQL
        return null;
    }
}

Analytics and Click Tracking Pipeline

A URL shortener is not just about redirection — it's a data business. Every click is valuable analytical data: geo-location, referrer, user agent, timestamp. You can't afford to write this data synchronously during a redirect (that would add latency). The pattern is asynchronous: the web server publishes a click event to a message queue (Kafka) and returns the 302/301 immediately. A separate consumer processes these events and updates the click_count in the database and aggregates data for dashboards.

Kafka topics can be partitioned by short key to maintain ordering per URL. The consumer can batch updates to the database (e.g., update click_count = click_count + 1 for 100 events at once). For real-time analytics, use a stream processor (Spark Streaming, Flink) to compute counters down to 1-minute granularity.

We also need to handle deduplication: users may refresh or multiple bots may click. Use a combination of IP + user agent + timestamp window to filter duplicates, or accept a small error percentage (most shorteners tolerate 1-2% overcount).

package io.thecodeforge.shortener;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class ClickEventPublisher {
    private static final Logger log = LoggerFactory.getLogger(ClickEventPublisher.class);
    private final KafkaProducer<String, String> producer;
    private final String topic = "url_clicks";

    public ClickEventPublisher(Properties kafkaProps) {
        this.producer = new KafkaProducer<>(kafkaProps);
    }

    public void publishClick(String shortKey, String userAgent, String ip, long timestamp) {
        String value = shortKey + "|" + userAgent + "|" + ip + "|" + timestamp;
        producer.send(new ProducerRecord<>(topic, shortKey, value), (meta, ex) -> {
            if (ex != null) log.error("Failed to publish click for " + shortKey, ex);
        });
    }

    public void close() {
        producer.close();
    }
}