URL Shortener Design — Why Auto-Increment Kills at Scale

Every time you see a link like 'bit.ly/3xQp9R' in a tweet, a QR code, or an SMS campaign, a surprisingly complex distributed system is working behind the scenes. URL shorteners process billions of redirects per day, and companies like Bitly, TinyURL, and Twitter's t.co have quietly become some of the most read-heavy services on the internet — often handling tens of thousands of requests per second at peak. Getting this design wrong at scale doesn't just mean slow pages; it means broken marketing campaigns, dead QR codes on printed packaging, and lost revenue that can't be recovered.

The core problem sounds trivial: map a long string to a short one and reverse the mapping on demand. But that simplicity is deceptive. You need to generate short codes that are globally unique, store hundreds of millions of mappings efficiently, serve redirects in under 10 milliseconds, handle hot keys (a single viral link getting millions of hits per minute), expire links, support custom aliases, and survive datacenter failures — all simultaneously.

By the end of this article you'll have a production-grade mental model for a URL shortener: you'll know exactly how to generate collision-free short codes, why you should never put a counter in a single database row, how to layer caching to absorb viral traffic spikes, and what the interview panel is really testing when they ask you this question.

What is Design URL Shortener?

A URL shortener is a service that takes a long URL and returns a shorter, unique alias that redirects clients to the original URL. The typical flow: a client submits a long URL via an API, the service generates a short code (e.g., 'abc123'), stores the mapping in a database with optional metadata (creation time, expiration, owner), and returns the full short URL (e.g., 'https://short.url/abc123'). When a client requests that short URL, the service looks up the code, retrieves the original URL, and issues an HTTP redirect (301 for permanent, 302 for temporary). Analytics (clicks, referrers, timestamps) are usually logged asynchronously.

Short Code Generation — Hashing vs Counter-Based IDs

There are two dominant strategies for generating short codes. The first is hash-based: take the long URL, compute a hash (e.g., MD5 or SHA-256), take the first N characters (usually 6–8), check for collisions, and if one exists add a salt or retry with a different prefix. The second is ID-based: use a globally unique integer (from a distributed ID generator) and encode it in base62 (0-9, a-z, A-Z) to produce a compact alphanumeric string. Base62 encoding of a 64-bit integer yields up to 11 characters — typical shorteners use 6–7 characters, which gives 62^6 ≈ 56 billion combinations.

ID-based systems are simpler for uniqueness (just generate a unique ID) but require a reliable ID generator. Hash-based systems must handle collisions and require longer codes for the same collision probability. Most production systems prefer ID-based with base62 encoding because the code space is deterministic and collision-checking is trivial.

package io.thecodeforge.shortener;

import java.util.ArrayList;
import java.util.List;

public class Base62Encoder {
    private static final String BASE62 = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
    private static final int BASE = 62;

    public static String encode(long id) {
        if (id == 0) return String.valueOf(BASE62.charAt(0));
        List<Character> chars = new ArrayList<>();
        while (id > 0) {
            chars.add(BASE62.charAt((int) (id % BASE)));
            id /= BASE;
        }
        StringBuilder sb = new StringBuilder(chars.size());
        for (int i = chars.size() - 1; i >= 0; i--) {
            sb.append(chars.get(i));
        }
        return sb.toString();
    }

    public static long decode(String shortCode) {
        long id = 0;
        for (char c : shortCode.toCharArray()) {
            int digit = BASE62.indexOf(c);
            if (digit == -1) throw new IllegalArgumentException("Invalid character in code: " + c);
            id = id * BASE + digit;
        }
        return id;
    }
}

Database Schema & Write Path

The core database stores the mapping from short code to long URL. The schema is simple: primary key on short_code, columns for original_url, created_at, expiration_at, owner_id (optional). But at scale, the write path must be designed for high throughput during creation bursts. Write operations are not the bottleneck (traffic is ~99% reads), but if you use a single database for ID generation, you get into trouble. Instead, decouple ID generation from the database: generate IDs in an application tier using Snowflake-like algorithms (or pre-allocated segments). Then insert the mapping asynchronously? No — inserts must be synchronous for consistency, but they can be batched and buffered.

For reads, index on short_code is critical. Use a covering index (include original_url) to avoid disk access. Partition the table by short_code prefix to distribute writes. Use a read replica for analytics queries, but always route redirect lookups to the primary or cache first.

-- TheCodeForge schema for URL shortener
CREATE TABLE links (
    short_code VARCHAR(10) NOT NULL PRIMARY KEY,
    original_url VARCHAR(2048) NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NULL,
    owner_id BIGINT NOT NULL DEFAULT 0,
    click_count BIGINT NOT NULL DEFAULT 0
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

-- Index for fast lookups (covering index)
CREATE INDEX idx_short_code ON links (short_code) INCLUDE (original_url, expires_at);

-- For analytics queries (non-urgent)
CREATE INDEX idx_owner_created ON links (owner_id, created_at);

Caching Layer — Survival Guide for Viral Traffic

A single viral link can generate millions of requests per minute. Without caching, your database will melt. The caching architecture needs at least two tiers: L1 (in-memory cache per application instance) and L2 (distributed cache like Redis or Memcached). L1 stores the hottest keys (recently accessed short codes) and evicts using LRU. L2 stores a larger set of mappings with a longer TTL.

Cache-aside pattern: on a redirect request, check L1 → if miss, check L2 → if miss, fetch from DB and populate both caches. Set a TTL of 24 hours for L2, but proactive invalidation when a link is deleted or expires. For read-heavy workloads, consider a write-through cache: on creation, immediately write to cache and DB asynchronously (with a queue). That way the first read is already fast.

Hot key problem: when a single short code gets 100k requests per second, Redis can become a hotspot. Solutions: local L1 caching (each app server caches the hot key), or use Redis with replicas and client-side sharding to distribute reads.

package io.thecodeforge.shortener;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import redis.clients.jedis.Jedis;

import java.util.concurrent.TimeUnit;

public class RedirectCache {
    private final Cache<String, String> l1 = CacheBuilder.newBuilder()
            .maximumSize(100_000)
            .expireAfterWrite(1, TimeUnit.MINUTES)
            .recordStats()
            .build();

    private final Jedis jedis;  // L2 connection pool
    private static final String PREFIX = "url:";
    private static final int L2_TTL_SECONDS = 86400;

    public String getOriginalUrl(String shortCode) {
        // L1 lookup
        String url = l1.getIfPresent(shortCode);
        if (url != null) return url;

        // L2 lookup
        url = jedis.get(PREFIX + shortCode);
        if (url != null) {
            l1.put(shortCode, url);
            return url;
        }

        // DB fallback happens outside this method
        return null;
    }

    public void put(String shortCode, String originalUrl) {
        l1.put(shortCode, originalUrl);
        jedis.setex(PREFIX + shortCode, L2_TTL_SECONDS, originalUrl);
    }
}

Redirect Mechanics — HTTP Status and Performance

When a client requests a short URL, the server must respond with an HTTP redirect. Two status codes matter: 301 (Moved Permanently) and 302 (Found). 301 tells the browser to cache the redirect permanently — subsequent requests go directly to the long URL without hitting the shortener. This is great for performance but breaks analytics if you want to count every click (because cached browsers don't hit your service). 302 tells the browser not to cache — every request hits the shortener, enabling click tracking.

Most services use 302 by default for dynamic analytics, and offer 301 as an option for permanent links. The redirect response also includes the Location header. The server must set CORS headers if the short URL is embedded in an iframe.

Performance: the entire redirect (from request to response) should complete in under 10ms at P99. This includes DNS resolution on the client side, TCP connection, TLS handshake, and the server processing. The server side is typically <1ms with caching. Server-side improvements: keepalive connections, HTTP/2 multiplexing, and edge caching (CDN).

HTTP/1.1 302 Found
Location: https://www.example.com/long-article-url
Cache-Control: no-cache, no-store, must-revalidate
Content-Length: 0
Access-Control-Allow-Origin: *

# Alternatively, a 301 redirect for permanent links:
HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/long-article-url
Cache-Control: public, max-age=31536000, immutable

Expiration, Custom Aliases, and Analytics

Real URL shorteners support link expiration (e.g., for temporary campaign links) and custom aliases (user picks a meaningful short code). Expiration is implemented by storing an expires_at column and checking during redirect lookup. If the current time exceeds expires_at, return 410 Gone or redirect to a fallback page. Custom aliases require a separate validation: they must be unique globally and not conflict with auto-generated codes. A common approach is to reserve a prefix for auto-generated codes (e.g., starting with a digit) and allow custom aliases to start with a letter. Or use two separate tables.

Analytics: every redirect should asynchronously log the click event (time, referrer, user-agent, IP) to a high-throughput queue (Kafka, Kinesis). A separate consumer processes the stream to update click counts and generate reports. The click count on the links table should be denormalised for quick display but must be updated asynchronously to avoid write contention. Use eventual consistency: the consumer updates the count in the DB via upsert.

-- TheCodeForge: check expiration during redirect lookup
SELECT original_url,
       CASE WHEN expires_at IS NOT NULL AND expires_at < CURRENT_TIMESTAMP THEN 1 ELSE 0 END AS expired
FROM links
WHERE short_code = ?
LIMIT 1;

-- Then in application code:
if (row.isExpired()) {
    response.setStatus(410); // Gone
    return;
}
redirectUserTo(row.getOriginalUrl());