Lesson 5.1• 8 min read

NoSQL data models and access patterns

Key-Value, Document, Wide-Column, and Graph stores compared against Relational RDBMS.

What you will learn

Say what NoSQL stores trade away compared with relational databases (BASE versus ACID).
Tell the key-value, document, wide-column and graph models apart, and what each suits.
Design a schema from the queries rather than from the entities.

The limits of relational modeling at extreme scale

Relational databases are strong at structured data with a strict schema, transactions, and complex joins, and they have clear, well-understood patterns for scaling. NoSQL stores make a different trade:

Data is denormalized, and joins are generally done in application code.
Most NoSQL stores lack true ACID transactions and favor eventual consistency. This is often summed up as BASE: basically available (the system guarantees availability), soft state (state may change over time, even without input) and eventual consistency. Compared with CAP, BASE chooses availability over consistency.

The primer's reasons to pick NoSQL: semi-structured data, a dynamic or flexible schema, non-relational data, no need for complex joins, many terabytes (or petabytes) of data, and very data-intensive workloads with very high IOPS. Data that suits it well: rapid ingest of clickstream and log data, leaderboards and scores, temporary data such as a shopping cart, frequently accessed ("hot") tables, and metadata or lookup tables.

The four major NoSQL data models

Model	Abstraction	Examples	Good for	Watch out for
Key-value	A hash table: usually O(1) reads and writes, often backed by memory or SSD; some keep keys sorted so key ranges are cheap	Redis, Riak, DynamoDB (key-value and document)	Simple data models and rapidly changing data, such as an in-memory cache layer	Only a limited set of operations, so complexity moves into the application
Document	A key-value store whose values are documents (JSON, XML, binary), queryable by their internal structure	MongoDB, CouchDB	Occasionally changing data with a flexible shape; documents in a collection can have completely different fields	The line with key-value stores blurs: many key-value stores also index value metadata
Wide column	Nested maps: `ColumnFamily<RowKey, Columns<ColKey, Value, Timestamp>>`; every value carries a timestamp for versioning and conflict resolution	Bigtable (the first), HBase, Cassandra	Very large data sets with high availability and scalability; keys kept in lexicographic order make key-range reads efficient	Queries that do not supply the row (partition) key
Graph	Nodes are records and arcs are relationships	Neo4j, FlockDB	Data with complex relationships, many foreign keys or many-to-many relationships, such as a social network	Relatively new and less widely used: fewer tools, and many are only reachable through REST APIs

Match each NoSQL data model to the job the lesson says it suits.

Key-valueDocumentWide columnGraph

Query-driven data modeling vs. relational normalization

In relational design you normalize data around entities (users, orders, products) and write the queries afterward, using joins.

In Cassandra you do the opposite: "you don't start with the data model; you start with the query model." You cannot join, so you design tables around the queries and denormalize freely. If an application needs orders by user and orders by status, you create two tables, each holding the data one query needs:

orders_by_user (user_id [partition], order_id [clustering], total, created_at)
orders_by_status (status [partition], created_at [clustering], order_id, user_id)

DynamoDB also starts from the access patterns, but AWS advises keeping as few tables as possible, often a single table, using sort keys and secondary indexes to serve the different queries. Either way, you must know the questions before you design the schema.

Figure 1: Specialized data models mapped across access patterns and workload characteristics.

Key takeaways

NoSQL stores denormalize, join in application code and mostly favor availability with eventual consistency (BASE).
Key-value suits simple, fast-changing data; document suits flexible shapes; wide column suits very large data read by key range; graph suits many-to-many relationships.
In Cassandra you start from the queries and may keep one table per query; DynamoDB also starts from the access patterns but keeps as few tables as possible.

Sources

Adapted from The System Design Primer by Donne Martin, licensed under CC BY 4.0. Changes were made.