Back of Envelope Estimation — The 5 Critical Numbers

Every large-scale system that ever failed probably had an engineer somewhere who skipped the math. Twitter's early 'fail whale' wasn't bad code — it was a system designed for far fewer requests than it actually received. Back-of-envelope estimation is the skill that separates engineers who build systems that survive launch day from those who scramble to add servers at 2 AM. It's not about being precise; it's about being right enough, fast enough, to make good architectural decisions.

Here's the hard truth: if you can't estimate storage within 2x and throughput within 3x before you write a line of code, you're flying blind. I've seen teams burn six-figure cloud bills because they forgot to multiply by replication factor. Estimation isn't the cost — it's the insurance.

The most common failure in production? Forgetting the peak factor. Your system handles 1000 QPS average, you provision for 1500. Then a viral tweet hits at 2 PM — traffic spikes to 8000 QPS. Your database melts. A 3x peak factor would have saved you. Estimate for peak, not average.

The Key Numbers Every Engineer Must Memorise

Back-of-envelope estimation relies on a small set of well-known numbers. Memorise these and you can size almost any system.

• Single server (commodity): ~10,000–20,000 QPS for simple API endpoints
• Database writes: ~1,000–5,000 writes/sec on a standard MySQL instance
• Disk read latency: ~2–10 ms for SSD, ~10–40 ms for HDD
• Network round trip in same region: ~0.5–2 ms
• Storage per user (social app): ~500 KB per user including profiles, photos thumbnails
• Storage per video minute: ~100 MB compressed (HD)
• Bandwidth per user for real-time chat: ~1 KB/s per active connection
• Caching (Redis) memory cost: ~$0.1/GB/month on cloud
• Database storage cost: ~$0.25/GB/month for SSD-backed cloud DB
• Egress cost (AWS): $0.09/GB to internet; $0.02/GB inter-region

These numbers change slowly and vary by cloud provider. Update them every six months. Use the cloud provider's pricing calculator for current rates.

Here's a trick: keep a personal cheat sheet in your notes app. When you run a load test or see a production bottleneck, update the constants. Over time, you'll build your own tuned set of numbers that are far more accurate than any generic list.

Don't trust cloud provider defaults blindly. I once saw an engineer use AWS's advertised RDS IOPS for a db.r5.large — actual throughput was less than half. Always baseline with your own load test.

Another trap: using 20k QPS for a Node.js server when your app does heavy JSON parsing. That number assumes simple requests. If each request deserialises a 50KB payload, your actual QPS drops to maybe 5k. Know your workload pattern.

The single most important number for cost modelling? Egress. Many teams model compute and storage perfectly, then forget data transfer. On AWS, 1 PB of egress costs $90,000. That's not a rounding error — it's a hiring decision.

// TheCodeForge — Estimation constants
// Use your own production baselines over generic numbers
const KEY_NUMBERS = {
  QPS_PER_SERVER: 20000,
  DB_WRITES_PER_SEC: 5000,
  DISK_READ_LATENCY_MS: 5,
  NETWORK_RTT_MS: 1,
  STORAGE_PER_USER_KB: 500,
  VIDEO_MINUTE_MB: 100,
  CHAT_BANDWIDTH_PER_USER_BPS: 1000,
  REDIS_COST_PER_GB_MONTH: 0.1,
  DB_STORAGE_COST_PER_GB_MONTH: 0.25,
  EGRESS_COST_PER_GB: 0.09
};

export function estimateStorage(totalUsers: number): number {
  return totalUsers * KEY_NUMBERS.STORAGE_PER_USER_KB;
}

How to Estimate Storage Requirements — Including Replication and Retention

Storage estimation is the most common use case and the most frequently miscalculated. Here's a step-by-step approach:

1. Define your data model — what entities do you store? e.g., user profile, posts, media, messages.
2. Estimate size per entity — average size of a user profile (name, bio, avatar) ~10 KB; a post with text and one image ~500 KB; a video minute ~100 MB.
3. Multiply by number of entities per user — e.g., each user creates 2 posts per day, receives 20 messages. Over retention period (30 days, 90 days, 1 year).
4. Include indexes and replication — indexes add 20–50% overhead; replication factor of 3 triples storage. B-tree indexes are larger than hash indexes; factor in index type.
5. Apply retention policy — data older than 90 days goes to cold storage. Hot storage costs 10x cold.
6. Add operational overhead — logs, transaction logs, backups. Add 50% overhead.

Example: 10 million users, each storing 100 MB over a year (including media uploads). That's 10e6 × 100 MB = 1 PB primary storage. With replication factor 3 → 3 PB. With indexes → 4 PB. With operational overhead → 6 PB. At $0.25/GB/month for hot storage, 6 PB = $1,500,000/month. That's not affordable for most. Back to the envelope: you need to reduce retention to 30 days for hot storage (80% reduction), move older data to cold ($0.004/GB → 98% cheaper), compress media (60% reduction), and use 2x replication for data that's not mission-critical.

Don't forget logs and backups. A common trap: estimate only user data, then discover that database transaction logs, application logs, and nightly backups triple the storage footprint. Add a 50% overhead for these operational data stores.

Another trap: assuming all users are equal. A small percentage of power users can generate 10x the average data. Run a percentile analysis if you can — P99 storage per user can be 5x the median.

Also consider storage tiering: hot data on SSD, warm on HDD, cold on object storage. Each tier has different cost and latency. Estimate access patterns to decide tier boundaries.

Real-world example: a photo-sharing app estimated 100KB per photo. But users uploaded in RAW format. Actual size was 5MB per photo. That 50x miss blew their storage budget in a week.

The retention trap: a team kept all user data forever. After 2 years, 80% of their storage cost was for data older than 90 days that nobody accessed. Moving old data to Glacier cut costs by 80% overnight.

# TheCodeForge — Storage estimation with overhead
class StorageEstimator:
    STORAGE_PER_USER_KB = 1024 * 100  # 100 MB over a year
    REPLICATION_FACTOR = 3
    INDEX_OVERHEAD = 1.3
    OPERATIONAL_OVERHEAD = 1.5  # logs, backups
    COST_PER_GB_MONTH = 0.25
    COLD_STORAGE_COST_PER_GB = 0.004  # Glacier Deep Archive
    HOT_RETENTION_DAYS = 30

    @classmethod
    def total_storage_gb(cls, users: int, retention_days: int = 365) -> float:
        return users * cls.STORAGE_PER_USER_KB / (1024 * 1024) * (retention_days / 365)

    @classmethod
    def cost_per_month(cls, users: int) -> float:
        # Hot storage for first 30 days, cold for remaining 335 days
        hot_storage = cls.total_storage_gb(users, cls.HOT_RETENTION_DAYS)
        cold_storage = cls.total_storage_gb(users, 365) - hot_storage
        
        raw_hot = hot_storage * cls.REPLICATION_FACTOR * cls.INDEX_OVERHEAD * cls.OPERATIONAL_OVERHEAD
        raw_cold = cold_storage * 2  # lower replication for cold data
        
        total = raw_hot * cls.COST_PER_GB_MONTH + raw_cold * cls.COLD_STORAGE_COST_PER_GB
        return total

print(f"10M users cost/month with tiering: ${StorageEstimator.cost_per_month(10_000_000):,.0f}")
print(f"Without cold tiering: ${StorageEstimator.cost_per_month(10_000_000) * 5:,.0f}")

How to Estimate Throughput (QPS and Bandwidth) — Peak Factor Is Not Optional

Throughput estimation tells you if your server fleet can handle the load. The most common mistake is using average QPS instead of peak.

1. Estimate total daily requests - Daily Active Users (DAU) × average actions per user per day = total daily actions - Example: 10M DAU, 10 actions/user/day = 100M actions/day

2. Convert to QPS - Average QPS = total daily actions / 86400 (seconds per day) - Peak QPS = average QPS × peak factor. Peak factor is 3-10x for consumer apps, 5-20x for e-commerce during sales. - QPS = 100M / 86400 * 3 (peak factor) ≈ 3,472 QPS

3. Estimate bandwidth - Each action involves data transfer (request + response). Assume average 50 KB per request. - Bandwidth = peak QPS × data per request = 3,472 × 50 KB ≈ 174 MB/s = ∼1.4 Gbps - Check if your network link and load balancer can handle 1.4 Gbps sustained.

4. Compare to server capacity - If each server handles 20k QPS, you need ceil(3,472 / 20,000) ≈ 1 server? Wait — that's for one service. For a microservice architecture, each service may handle part of the chain: web server, auth, business logic, database. Each has different capacity. - For a chain of 3 services, total required server count = sum of per-service QPS divided by per-server QPS.

A common mistake: forget that each request often triggers multiple internal requests. A single API call may hit auth, user service, and feed service, each adding to total QPS on downstream services. Account for fan-out (typical 3-5x).

Bonus: for write-heavy endpoints, DB writes are often the bottleneck before CPU. Estimate DB writes per second separately and compare against your DB's max transactional throughput (∼5k writes/sec for standard MySQL).

Also account for retry amplification: if a downstream service times out, the client may retry, multiplying load. Use circuit breakers to limit retry storms.

A real-world case: a payment service estimated 1000 TPS. But each payment call triggered 3 retries automatically when the downstream was slow. Actual load hit 3000 TPS, overwhelming the DB. Circuit breaker was the fix.

The peak factor mistake: using average QPS leads to crash under load. A social media app had 1000 QPS average, 3000 QPS peak. They provisioned for 1500 QPS. At 3 PM daily peak, the system collapsed. Adding a 3x peak factor would have saved them. Always multiply by 3-10x.

# TheCodeForge — Throughput estimation with fan-out factor
class ThroughputEstimator:
    FAN_OUT_FACTOR = 3  # each external action triggers 3 internal calls

    @staticmethod
    def qps(dau: int, actions_per_user: int, peak_factor: float = 3.0) -> tuple[float, float]:
        total_actions = dau * actions_per_user
        avg_qps = total_actions / (24 * 3600)
        external_peak_qps = avg_qps * peak_factor
        internal_peak_qps = external_peak_qps * ThroughputEstimator.FAN_OUT_FACTOR
        return external_peak_qps, internal_peak_qps

    @staticmethod
    def bandwidth(qps: float, bytes_per_request: int) -> float:
        bps = qps * bytes_per_request
        return bps / (1024**3) * 8  # Gbps

    @staticmethod
    def servers_needed(qps: float, qps_per_server: int = 20000) -> int:
        import math
        return math.ceil(qps / qps_per_server)

# Example
ext_qps, int_qps = ThroughputEstimator.qps(10_000_000, 10, peak_factor=3)
print(f"External peak QPS: {ext_qps:.0f}")
print(f"Internal peak QPS (with fan-out): {int_qps:.0f}")
print(f"Bandwidth (Gbps): {ThroughputEstimator.bandwidth(ext_qps, 50*1024):.2f}")
print(f"Servers needed for internal QPS: {ThroughputEstimator.servers_needed(int_qps)}")

How to Estimate Cache and Memory Requirements — Working Set vs Total Data

Caching is a critical lever for performance, but it's easy to underestimate the memory you'll need. The key concept: working set — the subset of data accessed frequently — not total data.

1. Estimate cache working set - Which data is accessed most frequently? User profiles, session tokens, product catalog, etc. - Size of one entry × number of unique entries accessed in a time window (e.g., 1 hour). - Example: 10M daily active users, each with a 1KB session token. But only 20% are active in any given hour = 2M users × 1KB = 2 GB working set for sessions.

2. Determine cache hit ratio target - Pareto principle: 80% of reads hit 20% of data. Estimate the hot set size. - If hot set is 2 GB and you dedicate 4 GB cache, you'll likely get >95% hit rate. Beyond that, diminishing returns. - For product catalog (read-only), working set can be 100% of data. For user sessions, working set is active users only.

3. Account for eviction overhead - Caches like Redis use eviction policies (LRU, LFU). Under memory pressure, evictions cause cache misses and increased DB load. - Rule: set memory limit to 1.5x your estimated working set to leave headroom for spikes.

4. Consider replication overhead - If you use Redis Cluster or a replicated cache, memory multiplies by replica count. - Example: 2 GB working set, replication factor 2 → 4 GB total cache memory.

5. Don't forget TTL overhead - Each cached entry has TTL metadata. With millions of keys, this adds 10-20% overhead to memory. - Use redis-cli --bigkeys to analyse actual memory usage vs estimates.

6. Cache access pattern matters - Read-heavy (90% reads): high hit ratio, memory is the constraint. - Write-heavy (90% writes): write-through cache, memory still helps but write amplification may occur.

A real-world trap: a team cached entire database rows instead of just hot columns. Their cache memory grew 3x beyond estimate, causing OOM and cascading failures.

Another gotcha: using a single Redis instance for both cache and session store. When one app's session floods evict the other app's cache. Separate your caches by use case.

The working set mistake: estimating total data instead of working set leads to 10x over-provisioning. For a 1TB database, the working set might be only 10GB (1%). Cache that, not the whole database.

# TheCodeForge — Cache memory estimation
class CacheEstimator:
    @staticmethod
    def working_set(active_users: int, size_per_entry_bytes: int, activity_pct: float = 0.2) -> float:
        """active_users = total users, activity_pct = fraction active in peak hour"""
        active_in_window = active_users * activity_pct
        return active_in_window * size_per_entry_bytes / (1024**3)  # GB

    @staticmethod
    def total_memory(working_set_gb: float, replica_factor: int = 2,
                     headroom: float = 1.5, metadata_overhead: float = 0.15) -> float:
        base = working_set_gb * headroom
        with_metadata = base * (1 + metadata_overhead)
        return with_metadata * replica_factor

# Example: 5M total users, 20% active per hour, 1KB session each
ws = CacheEstimator.working_set(5_000_000, 1024, activity_pct=0.2)
mem = CacheEstimator.total_memory(ws, replica_factor=2)
print(f"Working set (active users): {ws:.1f} GB")
print(f"Total cache memory needed: {mem:.1f} GB")

Estimating for Distributed Systems: Chain Latency and Fan-Out

In a microservice architecture, you don't just estimate a single endpoint — you estimate the whole chain. A user request might hit: - API Gateway - Authentication service - Business logic service - Database/cache Each hop adds latency, network bandwidth, and processing overhead.

How to estimate: 1. Identify the critical path: the longest chain of services for a request. 2. Estimate per-service QPS: external QPS × (fan-out factor per service). A single API call might fan out to 5 internal services (user lookup, inventory check, payment, notification, audit). 3. Estimate per-service bandwidth: QPS × request+response size for that service. 4. Estimate latency budget: service latency + network RTT. With 5 services + 5 network hops at 1ms each, that's 5× service latency + 5ms. 5. Add headroom: distributed systems degrade under load — add 30% overhead for retries, timeouts, and request amplification.

Example: Social feed request — Gateway (1ms + 0.5ms net) → Auth (5ms + 0.5ms net) → Feed service (20ms + 0.5ms net) → Cache (1ms) → DB (10ms). Total: ∼38ms for a single request. If you need to serve 10M users at peak 3k QPS, you need to ensure each service can handle its share of the QPS.

The chain latency number is critical for SLOs. If your SLO is 200ms P99, and your critical path estimate is 40ms average, you have room for tail latencies. But if the estimate is already 150ms, you're in trouble before you've built anything.

Also consider network bandwidth between services. If services are in different availability zones, bandwidth costs add up. Estimate inter-service traffic using average response sizes and QPS. At 10k QPS, 1KB per request = 10MB/s = 80Mbps, which is fine. At 100k QPS, it's 800Mbps — approaching 1Gbps limits.

The fan-out trap: assuming each external request generates one internal request. In reality, a single API call often generates 5-15 internal requests due to service decomposition. A team estimated 1000 QPS external, provisioned for 1000 QPS internal. Actual internal QPS was 10,000. The system collapsed.

A classic failure: an e-commerce team estimated 50ms for their checkout chain. In reality, each DB call took 30ms, and there were 4 DB calls. The chain was 120ms. They missed their P99 by 2x. They had to add caching and async processing post-launch.

# TheCodeForge — Distributed system estimation
class DistributedEstimator:
    @staticmethod
    def chain_latency(services: list[dict], headroom: float = 1.3) -> float:
        """services: list of {'service_latency_ms': float, 'network_rtt_ms': float}"""
        total = 0
        for svc in services:
            total += svc['service_latency_ms'] + svc['network_rtt_ms']
        return total * headroom

    @staticmethod
    def per_service_qps(external_qps: float, fan_out: dict[str, float]) -> dict:
        """fan_out: service_name -> multiplier (e.g., {'auth': 1, 'feed': 1, 'cache': 0.5})"""
        return {svc: external_qps * mult for svc, mult in fan_out.items()}

    @staticmethod
    def inter_service_bandwidth(qps: float, bytes_per_request: int) -> float:
        bps = qps * bytes_per_request
        return bps / (1024**3) * 8  # Gbps

# Example
chain = [{'service_latency_ms':1,'network_rtt_ms':0.5},
         {'service_latency_ms':5,'network_rtt_ms':0.5},
         {'service_latency_ms':20,'network_rtt_ms':0.5},
         {'service_latency_ms':1,'network_rtt_ms':0.2},
         {'service_latency_ms':10,'network_rtt_ms':0.2}]
print(f"Chain latency (ms): {DistributedEstimator.chain_latency(chain):.1f}")
print(f"Service QPS: {DistributedEstimator.per_service_qps(3472, {'auth':1, 'feed':1, 'cache':0.5})}")
print(f"Bandwidth between services at 3472 QPS, 10KB per call: {DistributedEstimator.inter_service_bandwidth(3472, 10*1024):.2f} Gbps")