Curriculum Progress & Saved Lab Designs

Chapter 1: Foundations of scalable systems

Core mental models: performance vs scalability, latency vs throughput, CAP theorem, and systematic scaling patterns.

Chapter 2: Availability, consistency and estimation

Master consistency models, failover architectures, exact mathematical availability calculations, powers of two, and latency magnitude estimation.

Chapter 3: The edge: routing, proxies and security

Master Anycast Geo-DNS, global CDN edge caching, Layer 4 vs Layer 7 load balancing, reverse proxy acceleration, and edge security/WAF mitigation.

Chapter 4: Scaling relational databases

Master primary-replica replication, replication lag mitigation, multi-master conflicts, database federation, horizontal sharding, consistent hashing, distributed transactions, and SQL query indexing.

Chapter 5: NoSQL and choosing a data store

Master NoSQL data models (Key-Value, Document, Wide-Column, Graph), B-Tree vs LSM-Tree storage engine internals, tunable quorum consistency ($R + W > N$), partition key skew mitigation, and polyglot persistence.

Chapter 6: Caching and invalidation

Master multi-tier caching hierarchies, cache update patterns (Cache-Aside, Write-Through, Write-Behind, Refresh-Ahead), stampede and thundering herd mitigation, and eviction algorithms (LRU, LFU, ARC).

Chapter 7: Asynchronous processing and events

Decouple synchronous request-response bottlenecks into resilient event-driven pipelines, compare task queues with distributed commit logs, guarantee delivery semantics with consumer idempotency keys, and handle queue lag and backpressure.

Chapter 8: Services, communication and consensus

Design high-performance service meshes with gRPC and Protocol Buffers, implement resilience patterns with circuit breakers, exponential backoff retries, and bulkheading, coordinate distributed transactions using Sagas, and establish distributed consensus with Raft and Paxos.

How to approach a design question

The System Design Primer’s four steps for a design question: use cases, constraints and estimates; a high level design; core components; and scaling the design, iteratively and with trade-offs.

Chapter 9: Capstone: Pastebin and Bitly

Design an enterprise-scale URL shortener and paste service with Base62 encoding, Key Generation Service (KGS), metadata vs object store decoupling, lazy vs active TTL expiration, hot link caching, and asynchronous analytics streaming.

Chapter 10: Capstone: Twitter timeline and search

The primer’s Twitter timeline and search: 100M active users, 6,000 tweets and 100,000 reads per second, fan-out on write versus on read and the hybrid, Redis timeline caching, Snowflake IDs, and Earlybird-style scatter-gather search.

Chapter 11: Capstone: a web crawler

The primer’s web crawler (1 billion links, 4 billion crawls a month), Mercator’s polite and prioritizing URL frontier, DNS caching, robots.txt, duplicate URL elimination, 64-bit SimHash near-duplicate detection, and WARC storage.

Chapter 12: Capstone: Mint.com

The primer’s Mint.com: 10M users, 5 billion transactions a month (2,000 writes per second), asynchronous extraction through a queue, idempotent ingestion, envelope encryption for account credentials, seller-based categorization and budgets.

Chapter 13: Capstone: social graph search

The primer’s social-graph search: 100M users, 50 friends each, shortest paths by BFS across Person Servers, bidirectional BFS, placing friends together, Facebook’s TAO cache, and ranking friend suggestions.

Chapter 14: Capstone: a distributed key-value cache

The primer’s query cache (10 billion queries a month, 270-byte entries, LRU and TTL), Memcached slab classes and calcification, consistent hashing and Redis Cluster hash slots, and replication with Sentinel or Cluster failover.

Chapter 15: Capstone: Amazon sales rank by category

The primer’s sales rank by category: 10M products, 1,000 categories, 400 purchases and 40,000 reads per second, an hourly MapReduce over the logs, optional stream processing, Redis sorted sets, and caching popular categories.

Chapter 16: Capstone: scaling on AWS

The primer’s step-by-step path from one server to millions of users on AWS, then what happens when an Availability Zone or a Region fails, and what the design costs.