CDN Caching — Why Your 24-Hour TTL Blocks Content Updates
A CDN purge returned success, but 200+ edge PoPs served stale images for 30 minutes.
20+ years shipping production systems from the metal up. Notes here come from systems that actually shipped.
- ✓Solid grasp of fundamentals
- ✓Comfortable reading code examples
- ✓Basic production concepts
- CDN stands for Content Delivery Network — a globally distributed network of edge servers.
- Edge servers cache static assets (images, CSS, JS) close to users to reduce latency.
- DNS routing directs each user to the nearest edge server based on geographic location.
- Cache TTL controls how long content stays fresh — too short increases origin load, too long causes stale content.
- Invalidation is hard: purging all edge servers takes time due to propagation delay.
- Biggest mistake: assuming cache headers are set correctly — one wrong header can bypass the entire CDN.
- Use X-Cache header to verify hit/miss status — if it's missing, your CDN may be bypassed entirely.
- Real challenge: cache key fragmentation from random query params can silently destroy hit ratios.
Imagine your favourite pizza place only has one kitchen in New York. If you order from Los Angeles, your pizza travels 2,800 miles — cold and late. Now imagine that pizza place opens mini-kitchens in every major city, each stocked with the most popular pizzas ready to go. That's a CDN. Instead of every user fetching files from one distant server, a CDN places copies of your content on dozens (or hundreds) of servers worldwide, so users always get served from the kitchen closest to them.
| Chrome | Firefox | Safari | Edge |
|---|---|---|---|
| ✓ | ✓ | ✓ | ✓ |
Every second of load time costs you users. Amazon famously found that a 100ms delay costs them 1% in sales. Netflix streams to 190 countries without melting a single origin server. Both rely on the same invisible infrastructure: Content Delivery Networks. CDNs are not just a performance luxury — for any application with a global or even national audience, they're table stakes.
The problem CDNs solve is simple but brutal: physics. Data travels through fibre optic cables at roughly two-thirds the speed of light. A user in Tokyo requesting an image hosted in Frankfurt will wait 150–200ms just for the round trip — before a single byte of content is transferred. Multiply that by dozens of assets per page and you've already lost the user. A CDN collapses that distance by caching content at geographically distributed edge servers so the round trip becomes 5–20ms instead.
By the end of this article you'll understand exactly what happens from the moment a browser requests a CDN-backed URL to the moment the content arrives. You'll know the difference between origin pull and push CDNs, how cache invalidation actually works (and why it's harder than it sounds), and how to configure cache headers so your CDN behaves exactly as you intend — not randomly. You'll also walk away with the mental models that senior engineers use when debugging CDN behaviour in production.
Here's a truth most engineers miss: your CDN is only as good as the weakest link in the chain. That could be a misconfigured Vary header, a stale DNS record, or a single line of code that sets Cache-Control: private. I've seen all three take down a production deployment. The goal of this guide is to make you the person who finds those weak links before they become incidents.
How CDN Caching Actually Works — And Why Your 24-Hour TTL Blocks Content Updates
A Content Delivery Network (CDN) is a globally distributed network of proxy servers that cache static and dynamic content closer to end users. The core mechanic is simple: when a user requests a resource, the CDN edge server serves a cached copy if the TTL (time-to-live) hasn't expired, otherwise it fetches a fresh copy from the origin. This reduces latency, offloads origin traffic, and improves availability. The TTL is set via HTTP headers like Cache-Control: max-age=86400 for 24 hours.
In practice, the CDN acts as a reverse proxy with a key-value cache. On a cache miss, the edge server forwards the request to the origin, caches the response, and returns it. On a cache hit, it returns the cached object without contacting the origin. The cache key is typically the full URL (including query parameters), but can be customized. Stale-while-revalidate and soft-purge allow serving stale content while fetching a new version in the background. Purge APIs invalidate cached objects by URL or tag, but propagation across all edge nodes takes seconds to minutes.
Use a CDN for any globally distributed application where latency matters — e.g., serving images, CSS, JS, API responses, or streaming video. It's essential for handling traffic spikes (like Black Friday) because the cache absorbs requests that would otherwise hit your origin servers. Without a CDN, a single server or load balancer becomes a bottleneck and single point of failure. The trade-off: you must design your caching strategy — TTL, cache keys, purge mechanisms — to balance freshness against hit rate.
How CDN Routing Works
When a user requests a CDN-backed URL, the browser first does a DNS lookup. The CDN's DNS server uses the user's IP to determine their geographic location and returns the IP of the nearest edge server. Some CDNs use anycast routing where the same IP is announced from multiple points and the internet's BGP routing chooses the closest. That's faster because it avoids an extra DNS hop. But it also means routing can drift if BGP paths change.
You can't control which edge a user hits — but you can test. Use tools like dig to see which CDN IP resolves for different DNS servers around the world. If you see a user in Brazil hitting a server in Texas, that's a routing problem worth investigating.
Edge servers don't just cache — they also terminate TLS, compress responses, and sometimes even execute edge-side includes. Every one of these features adds processing overhead, so you don't want to enable them all blindly. Measure before and after.
One production gotcha: DNS-based routing can misidentify users if their ISP uses a DNS resolver far from their actual location. Mobile users on 4G/5G may appear to be at the core network location, not their phone's location. Anycast avoids this but can cause asymmetric routing if BGP routes change.
For a deeper debugging routine: run curl -w '%{http_code} %{time_total} %{time_connect} %{time_starttransfer}' -o /dev/null -s https://yourdomain.com/file from a server in the affected region. A long time_connect suggests routing latency between user and edge. A long time_starttransfer may indicate origin response delay.
Another subtle point: CDN providers often have multiple tiers of routing — some use latency-based routing that measures real-time conditions via probes. That can shift traffic between PoPs dynamically, so a user's edge may change hour by hour. That's fine for static content but can cause issues for stateful edge compute. Plan accordingly.
Here's a practical way to test routing from multiple locations using a simple Python script:
Why Your DNS Isn't the Only Thing Routing Requests
Most devs think CDN routing is just DNS geolocation. It's not. When you request a file, your ISP's DNS resolver returns an IP based on your region — that's true. But the CDN's real routing happens at Layer 4 and 7. The edge server that gets your request doesn't just serve cached files. It runs a health check on every upstream path. If the nearest POP is under DDoS or has a failing disk, it routes you to the next-closest node — in milliseconds. That's why you see 30ms latency jumps during a regional outage: the network is actively failing over. The key: your request never hits the origin unless all edge nodes miss. That's why you can survive a datacenter fire. The CDN is a self-healing mesh. Treat it like one. Don't hardcode a single edge IP. That breaks the entire failover model.
Caching Is a Contract — You're Probably Breaking It
Your 24-hour TTL says 'this content is immutable for a day.' But you're updating that JS bundle every sprint. The CDN doesn't know that. It holds the old version until the TTL expires. That's not a bug — it's contract law. The Cache-Control header is a promise between you and the edge. If you set max-age=86400, you are legally obligated to not change the content for 24 hours. Break it? Users get stale assets. Fix: use content-addressed URLs. Hash your filenames. bundle.a1b2c3.js becomes unique per build. Now you can set a year-long TTL. Old versions expire naturally. The CDN never serves stale content. One pattern: include a version hash in the path — /v2/assets/ — not query strings. CDNs treat query params differently; some bypass cache entirely. Versioned paths are deterministic. Your cache hit ratio goes from 60% to 95%.
Edge Computing with CDNs: Cloudflare Workers, AWS Lambda@Edge
Edge computing extends CDN capabilities beyond caching by allowing you to run custom code at the edge—closer to your users. This reduces latency and offloads computation from your origin server. Two popular platforms are Cloudflare Workers and AWS Lambda@Edge.
Cloudflare Workers run JavaScript (or WebAssembly) on Cloudflare's global network. They can modify requests/responses, route traffic, or even build entire applications without a traditional server. For example, you can use a Worker to A/B test by rewriting URLs based on cookies:
```javascript // Cloudflare Worker: A/B test addEventListener('fetch', event => { event.respondWith(handleRequest(event.request)) })
async function handleRequest(request) { const url = new URL(request.url) const cookie = request.headers.get('Cookie') if (cookie && cookie.includes('variant=B')) { url.pathname = '/variant-b' + url.pathname } return fetch(url) } ```
AWS Lambda@Edge lets you run Node.js or Python functions in response to CloudFront events (viewer request/response, origin request/response). This is useful for dynamic header manipulation, URL rewrites, or authentication. For instance, you can add security headers to every response:
``javascript // Lambda@Edge: Add security headers exports.handler = (event, context, callback) => { const response = event.Records[0].cf.response; response.headers['x-frame-options'] = [{ key: 'X-Frame-Options', value: 'DENY' }]; response.headers['x-content-type-options'] = [{ key: 'X-Content-Type-Options', value: 'nosniff' }]; callback(null, response); }; ``
Edge computing is ideal for personalization, bot detection, and real-time transformations. However, be mindful of execution limits (e.g., Cloudflare Workers have 50ms CPU time per request on free plan) and cold starts with Lambda@Edge.
CDN Caching Strategies: TTL, Stale-While-Revalidate, Cache Invalidation
Effective caching requires choosing the right strategy for your content. The most common is Time-To-Live (TTL), which tells the CDN how long to keep a cached copy. A 24-hour TTL is simple but blocks updates. For dynamic content, shorter TTLs (e.g., 5 minutes) work better. However, you can do smarter.
Stale-While-Revalidate (SWR) allows the CDN to serve stale content while asynchronously fetching a fresh version. This eliminates the wait for revalidation. For example, with Cloudflare, you can set:
`` Cache-Control: public, max-age=300, stale-while-revalidate=86400 ``
This caches for 5 minutes, but for the next 24 hours, the CDN can serve the stale copy while updating in the background. Users never see a delay.
Cache invalidation is the process of removing cached content before TTL expires. Most CDNs offer purge APIs. For instance, with Cloudflare:
``bash curl -X POST "https://api.cloudflare.com/client/v4/zones/ZONE_ID/purge_cache" \ -H "Authorization: Bearer API_TOKEN" \ -H "Content-Type: application/json" \ --data '{"files":["https://example.com/path/to/file"]}' ``
But purging is slow (propagation delay) and costly (cache misses). Better to use versioned URLs (e.g., style.v2.css) or content hashing to force new cache entries. For APIs, use ETags and Last-Modified headers for conditional requests.
A practical example: a news site updates articles frequently. Use SWR with a short max-age (60s) and long stale-while-revalidate (1 hour). This ensures fresh content is served quickly while allowing background updates.
Multi-CDN Strategies for Global Applications
Relying on a single CDN can lead to regional outages or performance bottlenecks. A multi-CDN strategy uses multiple providers to improve reliability, performance, and cost. Common approaches include:
- DNS-based load balancing: Use a DNS service like Amazon Route 53 to route users to the fastest CDN based on latency or geolocation. Example: Route 53 latency records pointing to CloudFront and Cloudflare.
- Anycast with multiple providers: Some CDNs use Anycast (e.g., Cloudflare). You can combine them by having your DNS point to multiple Anycast IPs; the client will connect to the nearest.
- Active-Active with traffic splitting: Serve content from multiple CDNs simultaneously. For example, use Cloudflare for static assets and Fastly for dynamic APIs. This requires careful session management.
- Failover: Use one CDN as primary and another as backup. Monitor health checks and switch via DNS or a load balancer.
Implementation example with DNS-based routing:
``yaml # Route 53 configuration records: - name: www.example.com type: A alias: true alias_target: dns_name: d123.cloudfront.net zone_id: Z2FDTNDATAQYW2 set_identifier: CloudFront failover: PRIMARY - name: www.example.com type: A alias: true alias_target: dns_name: cdn.example.com # Cloudflare proxy zone_id: Z2FDTNDATAQYW2 set_identifier: Cloudflare failover: SECONDARY ``
Challenges include cache synchronization (e.g., purging across all CDNs), SSL certificate management, and cost tracking. Use a CDN management platform like Cedexis or internal tooling to orchestrate.
A real-world example: Netflix uses multiple CDNs (including their own Open Connect) to deliver video. They use DNS-based routing to direct users to the best performing CDN node.
The Great Image Refresh Failure — 24-Hour Stale Cache Nightmare
- Always know your CDN's default TTL for each asset type.
- Assume a cached asset will stay cached until explicitly invalidated.
- Use versioned URLs or fingerprinting to force cache refresh on content change.
- Always verify purge propagation — don't trust the API response alone.
- Implement monitoring for cache invalidation completion to catch partial failures.
- Automate cache invalidation tests in your CI/CD pipeline — simulate a user request from multiple regions to confirm old content is gone.
- Set up an alert when purge takes longer than the provider's SLA (typically 5 minutes).
curl -I https://example.com/image.png | grep -i 'x-cache'curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://example.com/image.png| File | Command / Code | Purpose |
|---|---|---|
| cdn_routing_test.py | locations = [ | How CDN Routing Works |
| check-cdn-routing.sh | curl -v -o /dev/null -s https://cdn.thecodeforge.io/assets/bundle.js \ | Why Your DNS Isn't the Only Thing Routing Requests |
| cdn-cache-buster.go | "crypto/sha256" | Caching Is a Contract |
| cloudflare-worker-ab-test.js | addEventListener('fetch', event => { | Edge Computing with CDNs |
| cdn-cache-headers.conf | location /api/ { | CDN Caching Strategies |
| route53-multi-cdn.yaml | records: | Multi-CDN Strategies for Global Applications |
Key takeaways
Vary header, `Cache-ControlInterview Questions on This Topic
You purge a cached image and the API returns 200, but users still see the old version. Walk through your debugging steps.
Age and CF-Cache-Status or equivalent headers from multiple locations to see if specific edges are still hitting cache. Use dig and curl from different regions to identify if routing is sending users to unpurged PoPs. Verify the purge scope — URL vs tag vs wildcard — matched the actual cache key including query parameters. Check if origin shield or a secondary tier is caching independently. Finally, confirm the original response headers didn't include Cache-Control: private or a Vary header that would fragment the cache.Frequently Asked Questions
20+ years shipping production systems from the metal up. Notes here come from systems that actually shipped.
That's Computer Networks. Mark it forged?
7 min read · try the examples if you haven't