FjellSystem Design Lab
Lesson Method 4• 5 min read

Step 4: Scale the design

Benchmark, profile, fix the bottleneck the numbers point to, and repeat; every fix is a trade-off.

What you will learn

  • Scale by iterating: benchmark, profile, address the bottleneck, repeat.
  • Let step 1's numbers decide how far scaling needs to go.
  • Say what problem each scaling change solves, and its alternatives.

Step 4: find the bottleneck, then fix it

The last step is to identify and address bottlenecks, given the constraints. The primer lists the usual tools (a load balancer, horizontal scaling, caching, database sharding) and asks whether you need them, not whether you can name them. Discuss the possible solutions and their trade-offs: in the primer's words, everything is a trade-off.

Iterate, do not jump

Seven of the eight worked solutions carry the same warning: do not simply jump from the initial design to the final one. (The AWS solution instead shows the iteration, stage by stage.) Say that you would scale iteratively:

  1. Benchmark or load test.
  2. Profile to find the bottleneck.
  3. Address it, weighing the alternatives and their trade-offs.
  4. Repeat.

For each change, be ready to say what problem it solves. The primer's prompts: what does adding a load balancer with several web servers address? A CDN? Read replicas? What are the alternatives and the trade-offs of each?

Scaling Pastebin from its numbers

Step 1's estimates decide how far step 4 needs to go. In the primer's Pastebin solution:

  • Reads, 40 a second on average and higher at peak: popular pastes are served from a memory cache instead of the database, which also absorbs uneven traffic and spikes. SQL read replicas handle the cache misses, as long as they are not bogged down replicating writes.
  • Writes, 4 a second on average: a single SQL primary with a standby should manage. Only if not would you reach for federation, sharding, denormalization or SQL tuning.
  • Storage, 12.7 GB of new content a month: an object store such as Amazon S3 handles this comfortably.
  • Analytics move to a data warehouse such as Amazon Redshift or Google BigQuery.

Notice what is not in the list: no sharding and no queue. The numbers do not call for them, and adding them anyway would cost complexity for nothing.

Then keep going

With time left, the primer suggests talking points: whether some data belongs in NoSQL, where and when to cache, asynchronism, how services communicate, and security. And scaling is never finished: keep benchmarking and monitoring, and address bottlenecks as they appear.

The lab in this course works the same way. Its calculation panel shows each hop's utilization, so the tier to fix first is the busiest one, and its findings name it.

Simplified from the primer’s scaled Pastebin (which also has DNS, a CDN and an analytics database): a memory cache and read replicas for reads, one primary for 4 writes a second, an object store for the text.
  • Client: 1 × Clients & Users
  • Load balancer: 2 × Application Load Balancer
  • Write API: 2 × App Compute Server
  • Read API: 3 × App Compute Server
  • Memory cache: 2 × Redis In-Memory Cache
  • SQL primary and standby: 2 × Primary Relational DB
  • SQL read replicas: 2 × Database Read Replica
  • Object store: 1 × Object Storage (S3)
  • Client sends requests to Load balancer
  • Load balancer sends requests to Write API
  • Load balancer sends requests to Read API
  • Write API sends requests to SQL primary and standby
  • Write API sends requests to Object store
  • Read API sends requests to Memory cache
  • Memory cache sends requests to SQL read replicas
  • Read API sends requests to Object store

Key takeaways

  • Do not jump from the first sketch to the final design; show each change and the bottleneck it removes.
  • For Pastebin, a cache and read replicas handle reads, one primary with a standby handles 4 writes a second, and an object store handles the content.
  • No sharding and no queue: the numbers do not call for them, and adding them costs complexity for nothing.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.