FjellSystem Design Lab
Lesson 3.2• 6 min read

Content delivery networks (CDN)

Edge points of presence (POPs), push vs pull CDNs, HTTP cache headers, and speeding up dynamic requests.

What you will learn

  • Explain what a CDN saves, and what decides how much.
  • Choose between a pull CDN and a push CDN.
  • Control caching with Cache-Control headers, and see how a CDN speeds up responses it cannot cache.

What is a Content Delivery Network?

A content delivery network (CDN) is a globally distributed network of proxy servers (points of presence, or POPs) that serve content from locations close to users. CDNs usually serve static files (HTML, CSS, JavaScript, images, video); some, such as Amazon CloudFront, also handle dynamic content. The site's DNS tells clients which server to contact.

A CDN helps in two ways: users get content from a nearby data center, and your own servers do not have to serve the requests the CDN answers. How much it helps depends on your traffic: the share of requests that are cacheable and the hit ratio the CDN achieves.

Only the misses reach your origin, and they cost bandwidth as well as requests: the origin must send misses per second × object size. At 1,000 misses a second of 250 KB each that is 250 MB/s, or 2 Gbit/s (8 bits to a byte), twice what a 1 Gbit/s link carries. A higher hit ratio cuts both.

Push CDN versus pull CDN

Topology How it works Pros Cons Works well for
Pull CDN The CDN fetches content from your server the first time a user requests it, then caches it for a TTL. Minimal storage on the CDN; only recently requested content stays there. The first request for each object is slower; objects that expire and are re-pulled before they change cause redundant traffic. Sites with heavy traffic, which spreads requests evenly.
Push CDN You upload content to the CDN whenever it changes, and rewrite URLs to point at the CDN. Content is uploaded once, only when new or changed; you control when it expires. Maximizes storage; you are responsible for uploading everything. Sites with little traffic, or content that rarely changes.

Either way, CDN costs can be significant (weigh them against serving the traffic yourself), content can be stale until its TTL expires, and static URLs must be rewritten to point at the CDN.

HTTP Cache-Control headers

Caching is controlled with standard HTTP headers:

HTTP/1.1 200 OK
Content-Type: image/webp
Cache-Control: public, max-age=31536000, immutable
ETag: "68b329da9893e34099c7d8ad5cb9c940"
  • public: Explicitly marks the response as cacheable, including by shared caches such as a CDN.
  • max-age=<seconds>: How long the response stays fresh, for any cache (browser or CDN).
  • s-maxage=<seconds>: Freshness for shared caches (a CDN) only; overrides max-age there.
  • stale-while-revalidate=<seconds>: After the response goes stale, a cache may keep serving it for this long while it revalidates in the background.
  • immutable: The server will not change this resource while the response is fresh, so clients need not revalidate it, even on reload. It is typically used on URLs that contain a hash of the content, so a new version gets a new URL.

Speeding up dynamic requests

Even responses that cannot be cached can get faster through a CDN:

  • Short handshakes: The user's TCP and TLS handshakes complete with a nearby edge location rather than with a distant origin, so each handshake round trip is short.
  • Reused origin connections: After a response, the edge keeps its connection to the origin open for a few seconds (CloudFront documents this), so subsequent requests skip a new TCP connection and TLS handshake.

Edge cache offload and origin shielding

Edge hit ratio
88%
Edge p99 latency
8 ms
Origin egress saved
39.6 GB
Recent edge request stream:
/media/hero-banner.webpHIT (Edge)Edge POP8 ms
/assets/app.chunk-5a8b.jsHIT (Edge)Edge POP6 ms
/api/v1/user/checkoutMISS (Origin)Origin app server64 ms
/media/avatar-9921.pngHIT (Edge)Edge POP9 ms

Pull CDN: On the first user request for an asset, the edge POP incurs a cache miss and pulls the asset from the origin (e.g. S3 or app server). Later requests at that location are served from the edge cache until the TTL expires.

Figure 2: Edge POP offloading static media to S3 and forwarding dynamic API misses to Origin ALBs.
  • Global Users: 1 × Clients & Users
  • CloudFront Edge POPs: 4 × CloudFront / CDN Edge
  • S3 Media Storage: 2 × Object Storage (S3)
  • Origin ALB: 2 × Application Load Balancer
  • App Servers: 6 × App Compute Server
  • Global Users sends requests to CloudFront Edge POPs
  • CloudFront Edge POPs sends requests to S3 Media Storage
  • CloudFront Edge POPs sends requests to Origin ALB
  • Origin ALB sends requests to App Servers

Key takeaways

  • Only misses reach the origin, and each costs bandwidth: misses a second × object size. The cacheable share and the hit ratio set the savings.
  • A pull CDN fetches on the first request and keeps little; a push CDN holds what you upload, and you control when it changes.
  • max-age, s-maxage, stale-while-revalidate and immutable (on URLs that contain a content hash) decide how long copies stay fresh.
  • Uncacheable requests still gain: handshakes end at a nearby edge, which reuses its connection to the origin.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.