Lesson 10.1• 9 min read

Requirements, estimates and the timeline API

The primer’s use cases and estimates: 100M active users, 6,000 tweets and 100,000 reads per second, 150 TB of new content a month, and the REST API.

What you will learn

State the Twitter timeline and search use cases and assumptions as the primer does.
Estimate tweet storage, fan-out deliveries and read rates.
Size the home timeline cache, including its replicas.

The problem

The primer's Design the Twitter timeline and search notes that designing the Facebook feed and Facebook search are similar questions.

Use cases (from the primer)

A user posts a tweet; the service pushes it to followers, sending push notifications and emails.
A user views the user timeline (their own activity) and the home timeline (activity from people they follow).
A user searches keywords.
The service has high availability.

Out of scope: the firehose and other streams, visibility rules (such as hiding @replies) and analytics.

Assumptions (from the primer)

Traffic is not evenly distributed. Posting should be fast, and so should fanning out to followers, "unless you have millions of followers".
Viewing the timeline and searching should be fast. Twitter is read-heavy, so optimize for fast reads; ingesting tweets is write-heavy.
100 million active users.
500 million tweets per day (15 billion a month), each delivered to 10 followers on average: 5 billion deliveries a day, 150 billion a month.
250 billion read requests and 10 billion searches per month.

Back-of-the-envelope estimates

Using the primer's conversion, 400 requests per second ≈ 1 billion requests per month:

Quantity	Per month	Per second
Read requests	250 billion	$250 \times 400 = 100{,}000$
Tweets	15 billion	$15 \times 400 = 6{,}000$
Fan-out deliveries	150 billion	$150 \times 400 = 60{,}000$
Searches	10 billion	$10 \times 400 = 4{,}000$

Size per tweet (primer): tweet_id 8 bytes, user_id 32 bytes, text 140 bytes, media 10 KB on average. That is about 10 KB per tweet.

10\text{ KB} \times 500 \times 10^6 \times 30 = 150\text{ TB of new content per month}

150\text{ TB} \times 36 = 5.4\text{ PB in 3 years}

For comparison, Twitter reported 150 million active users and about 5,000 tweets per second on average (7,000 at the daily peak) in 2013, and 300,000 queries per second to generate timelines. So the primer's figures are the right order of magnitude.

Sizing the timeline cache

The primer keeps home timelines in a memory cache such as Redis. Each entry is a tweet ID, a user ID and a metadata byte: $8 + 8 + 1 = 17$ bytes. It keeps "only several hundred tweets for each home timeline", and only for active users. Twitter capped home timelines at 800 entries. With the primer's 100 million active users:

100 \times 10^6 \times 800 \times 17\text{ B} \approx 1.36\text{ TB}

Twitter reported "a couple of terabytes of RAM" for this cluster, and said each tweet is replicated on 3 machines. Replication multiplies the RAM you need, so size the cluster for the copies too.

API

The primer's REST API:

POST /api/v1/tweet
{ "user_id": "123", "status": "hello world!", "media_ids": "ABC987" }

→ { "created_at": "Wed Sep 05 00:37:15 +0000 2012", "status": "hello world!",
    "tweet_id": "987", "user_id": "123", ... }

GET /api/v1/home_timeline?user_id=123

→ [ { "user_id": "456", "tweet_id": "123", "status": "foo" }, ... ]

The user timeline API is similar, and search is GET /api/v1/search?query=hello+world. The primer's request also carries an authentication credential, omitted here. For internal calls it suggests RPC.

Figure 1: High-level Twitter system architecture separating write fan-out ingestion from low-latency timeline feed reads.

Key takeaways

Twitter is read-heavy, so optimize for fast timeline reads; ingesting tweets and fanning them out is write-heavy.
500 million tweets a day at about 10 KB each is 150 TB of new content a month, 5.4 PB in three years.
800 entries of 17 bytes for 100 million active users is about 1.36 TB of cache, before replication multiplies it.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.