How to Answer System Design Interview Questions (Step-by-Step Framework)

// ── PHASE 1: REQUIREMENT CLARIFICATION SCRIPT ──────────────────────────────
// Use this verbatim in your next interview. Adapt the numbers to the prompt.

// STEP 1 — Functional Scope
"Before I start designing, I want to make sure we're aligned on scope.
 Let's say we're building Twitter's core. I'm going to focus on:
   (1) A user posting a tweet
   (2) A user reading their home timeline
 Out of scope for now: search, DMs, ads, trending topics.
 Does that work for you, or would you like me to reprioritize?"

// STEP 2 — Scale Estimation (do this out loud, show your work)
"Let me size this system:
  - 500M daily active users
  - Assume 20% post content: ~100M tweets/day
  - 100M tweets / 86,400 seconds ≈ 1,200 tweet writes/second (peak ~3x = 3,600 wps)
  - Read:Write ratio is typically 100:1 for social feeds
  - So reads: ~120,000 reads/second (peak ~360,000 rps)
  - Tweet storage: avg 200 bytes per tweet * 100M/day = 20GB/day, ~7TB/year
  - Media (images/video) stored separately in object storage — not in the tweet DB"

// STEP 3 — Non-Functional Requirements
"A few more constraints I want to nail down:
  - Latency: home timeline should load in under 200ms p99
  - Availability: 99.99% uptime (four nines) — no scheduled downtime
  - Consistency: eventual consistency is OK for the timeline
    (a user seeing a tweet 2 seconds late is fine)
  - Durability: tweets must never be lost once written
  - Geo: assume global, so we need multi-region support"

// OUTPUT: You now have a design contract. Everything you build next
// is justified against these numbers. This is what separates architects
// from coders.

// ── PHASE 2: HIGH-LEVEL DESIGN WALKTHROUGH ──────────────────────────────────
// Narrate this while drawing on the whiteboard. Never draw in silence.

// STEP 1 — API Contract (define before backend)

API DESIGN — Tweet Posting Flow:
  POST /v1/tweets
  Request:  { user_id: UUID, content: string(280), media_ids: UUID[] }
  Response: { tweet_id: UUID, created_at: ISO8601, status: 'published' }
  
  // Why cursor pagination on the timeline read?
  // Offset-based: SELECT * FROM tweets LIMIT 20 OFFSET 10000
  //   → DB scans 10,020 rows and discards 10,000. O(n) cost. Breaks at scale.
  // Cursor-based: SELECT * FROM tweets WHERE id < :last_seen_id LIMIT 20
  //   → Uses the index directly. O(log n). Stable even when rows are inserted.

API DESIGN — Home Timeline Read:
  GET /v1/users/{user_id}/timeline?cursor={opaque_cursor}&limit=20
  Response: {
    tweets: [ { tweet_id, author_id, content, media_url, created_at, like_count } ],
    next_cursor: "eyJ0d2VldF9pZCI6IjEyMzQ1NiJ9"  // base64-encoded last tweet ID
  }

// STEP 2 — Service Decomposition

  [Mobile / Web Client]
       ↓ HTTPS
  [CDN — static assets, cached timeline responses]
       ↓ cache miss
  [API Gateway — rate limiting, auth token validation, routing]
       ↓
  ┌──────────────────────────────────────────────┐
  │  Tweet Service    Timeline Service           │
  │  User Service     Notification Service       │
  │  Media Service    Search Service (out-scope) │
  └──────────────────────────────────────────────┘
       ↓                    ↓
  [Tweet DB]         [Timeline Cache]
  [User DB]          [Fan-out Queue]
  [Object Store]     [Notification Queue]

// STEP 3 — Call out the coupling anti-pattern proactively
"One thing I want to be explicit about: each service owns its own datastore.
 The Timeline Service does NOT query the Tweet Service's database directly.
 Communication between services is via APIs or async events on a message queue.
 Shared databases create hidden coupling that makes services impossible to
 scale or deploy independently — I've seen this cause outages in production."

// ── PHASE 3: FAN-OUT DEEP DIVE — THE HARD PROBLEM ──────────────────────────

// HYBRID FAN-OUT ALGORITHM (production-grade)

When a tweet is posted:
  1. Tweet Service writes tweet to tweets DB (source of truth)
  2. Tweet Service publishes event to Fan-out Queue (Kafka topic: 'new-tweets')

Fan-out Worker picks up the event:
  3. Fetch author's follower list from User Service (graph DB or followers table)
  4. Check follower count:
     IF author.follower_count < CELEBRITY_THRESHOLD (e.g., 10,000):
       → PUSH path: for each follower_id, write tweet_id to their Redis timeline
           ZADD timeline:{follower_id} {timestamp_score} {tweet_id}
           // Redis ZSET keeps timeline sorted by time automatically
           // Trim to last 800 entries to bound memory:
           ZREMRANGEBYRANK timeline:{follower_id} 0 -801
     ELSE:
       → PULL path: mark tweet as 'celebrity-tweet', store only in tweets DB
           No fan-out write. Will be fetched at read time.

When a user requests their timeline:
  5. Timeline Service reads precomputed timeline from Redis:
       ZREVRANGE timeline:{user_id} 0 19  // last 20 tweet_ids, newest first
  6. Identify which users the requester follows AND have celebrity status:
       celebrity_follows = graph.get_celebrity_follows(user_id)
  7. For each celebrity, fetch their latest tweets from tweets DB:
       SELECT tweet_id FROM tweets
       WHERE author_id = :celebrity_id
       ORDER BY created_at DESC LIMIT 20
  8. MERGE precomputed timeline + celebrity tweets in memory:
       merged = sort_by_time(redis_tweets + celebrity_tweets)[:20]
  9. Hydrate tweet_ids → full tweet objects via Tweet Service (batch GET)
  10. Return hydrated, merged, paginated timeline to client

// WHY THIS WORKS:
// - Normal users (99.9% of authors): write amplification is bounded
//   1,000 followers * 3,600 writes/sec = 3.6M Redis ops/sec — achievable
// - Celebrities: no write amplification. 50M followers * 0 writes = 0
// - Read path: one Redis lookup + a handful of DB queries for celebrities
//   This keeps p99 latency under 200ms even at scale

// EDGE CASES TO CALL OUT IN YOUR INTERVIEW:
// 1. User follows transition: if @JohnDoe gains 10,001 followers, backfill
//    existing timelines? No — just switch the threshold prospectively.
//    Slight inconsistency window is acceptable (eventual consistency).
// 2. Redis eviction: if a user hasn't logged in for 90 days, their timeline
//    key expires. On return, trigger an async cold-start timeline rebuild.
// 3. Follower list pagination: fetching 50M follower IDs in one call will OOM
//    your fan-out worker. Stream the follower list in batches of 10,000
//    using cursor pagination against the graph store.
// 4. At-least-once delivery: Kafka guarantees at-least-once. A fan-out worker
//    crash mid-batch could re-process the event. Make ZADD idempotent —
//    re-adding the same member to a ZSET with the same score is a no-op.

// ── PHASE 4: BOTTLENECK REVIEW AND CLOSING STATEMENT ────────────────────────

// PROACTIVE BOTTLENECK IDENTIFICATION (say this before they ask)

"Let me stress-test my own design before we wrap up."

// BOTTLENECK 1 — Fan-out Queue Lag
"If we get a sudden traffic spike — say, a major world event drives 10x normal
 tweet volume — our Kafka consumer group (fan-out workers) may fall behind.
 The symptom: users see stale timelines. The mitigation:
  - Monitor consumer lag per Kafka partition (alert if lag > 30 seconds)
  - Auto-scale fan-out worker pods based on lag metric (HPA in Kubernetes)
  - For extreme outliers, shed load: if fan-out lag exceeds 5 minutes,
    temporarily switch all users to fan-out-on-read as a degraded mode"

// BOTTLENECK 2 — Redis Memory at Scale
"500M users * 800 tweet_ids * 8 bytes each = ~3.2TB of Redis storage.
 That's expensive but achievable with a Redis cluster.
 Optimizations:
  - Expire timeline keys for users inactive > 30 days (EXPIRE command)
  - Store only tweet_id (int64, 8 bytes) not full tweet objects
  - Use Redis cluster with consistent hashing to distribute keys
  - Monitor memory fragmentation ratio — above 1.5 means Redis is
    allocating but not efficiently using memory; trigger MEMORY PURGE"

// BOTTLENECK 3 — Single-Region Failure
"My design so far is single-region. For 99.99% availability globally,
 we need geo-redundancy:
  - Deploy identical stacks in 3 regions: US-East, EU-West, AP-Southeast
  - Use latency-based DNS routing (Route 53 / Cloudflare) to pin users
    to their nearest region
  - Tweets are written to the local region and asynchronously replicated
    to other regions via Kafka MirrorMaker 2 with ~500ms replication lag
  - Conflict resolution: last-write-wins with tweet_id as tiebreaker
    (safe because tweet creation is idempotent)"

// CLOSING SUMMARY (always do this — most candidates skip it)
"To summarize what we built:
  - Core entities: tweets, users, follower graph, precomputed timelines
  - Write path: Tweet Service → Kafka → Fan-out Workers → Redis + Tweet DB
  - Read path: API Gateway → Timeline Service → Redis (ZSET) + celebrity DB queries
  - Key trade-off: hybrid fan-out adds operational complexity but is the only
    approach that keeps both write amplification and read latency bounded
  - Scale: designed for 1,200 wps, 120,000 rps, 7TB/year tweet storage
  - Reliability: 99.99% via multi-region active-active with async replication"

// YOUR QUESTION TO THE INTERVIEWER
"One thing I'd love to know — how does your team actually handle
 the fan-out problem today? Did you go hybrid, or take a different approach?"