Lesson Method 3• 5 min read

Step 3: Core components

Take the use cases one at a time and go deep where the problem is hardest: data model, key generation, API.

What you will learn

Choose which components to take deep, and say which stay shallow.
Work one component through in detail: generating short links for Pastebin.
Write the data model and API from what step 1 sized.

Step 3: go deep where it matters

The third step is to dive into the details of each core component. The primer's own example is the short-link service, and for it, it lists what to discuss:

Generating and storing a hash of the full URL: MD5 and Base62, hash collisions, SQL or NoSQL, and the database schema.
Translating a hashed URL to the full URL: the database lookup.
The API and object-oriented design.

Two habits make this step work. Take the use cases one at a time, as the worked solutions do (each has a heading "Use case: …"). And ask how much code you are expected to write: the primer repeats that advice in several solutions.

A worked component: the short link

In the Pastebin solution, the Write API makes a link by taking the MD5 hash of the user's IP address and the time, encoding it in Base62 (the characters a–z, A–Z and 0–9, which need no escaping in a URL), and keeping the first 7 characters. Seven Base62 characters allow

62^{7} \approx 3.5 \times 10^{12}

values, far more than the 360 million links expected over three years. Base64 is avoided because its + and / characters cause trouble in URLs.

The link is the primary key of the pastes table, so the database itself rejects a duplicate, and the Write API checks for one and generates another if needed. A second index on created_at speeds up lookups by time.

The data model and the API

The table needs only what step 1 sized: the short link, the expiry, the creation time and the path to the text in the object store. The text itself is in the object store.

The public API is REST: a POST with the paste's contents and expiry returns the short link; a GET with the short link returns the contents. Internal calls between services could use RPC. The read path checks the database for the link, fetches the text from the object store if it is there, and returns an error if not.

Depth, not breadth

You will not have time to detail every box. Choose the components where the use cases are hardest or the numbers are largest, and say which ones you are leaving shallow and why. In Pastebin that is link generation and the read path; analytics can stay a one-line batch job because the assumptions allow it.

The write path in detail: the Write API makes a 7-character Base62 link, stores a row keyed by it, and puts the text in the object store.

Key takeaways

Take the use cases one at a time, and ask how much code you are expected to write.
Seven Base62 characters give $62^7 \approx 3.5 \times 10^{12}$ links, far more than needed; making the link the primary key lets the database reject duplicates.
The table holds only the small fields step 1 sized; the text lives in the object store.
Go deep where the use cases are hardest or the numbers largest.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.