Partition keys, skew and polyglot persistence

Primary key anatomy in distributed NoSQL

In Cassandra (and, with different names, DynamoDB), the primary key has two parts:

1. Partition key: Hashed to pick the nodes that own the partition (Cassandra's default partitioner is Murmur3).
2. Clustering key: Sets the sort order of rows within that partition, so ranges within one partition are cheap to read.

-- Cassandra CQL Example:
CREATE TABLE sensor_readings (
    device_id uuid,          -- Partition Key (Determines Shard)
    recorded_at timestamp,   -- Clustering Key (Determines On-Disk Sort)
    temperature double,
    humidity double,
    PRIMARY KEY (device_id, recorded_at)
) WITH CLUSTERING ORDER BY (recorded_at DESC);

The Hot Partition & Skew Catastrophe

If partition keys are poorly chosen, all writes or reads will concentrate on a single physical shard, neutralizing horizontal scaling:

1. The time-bucket hot partition

• Mistake: Using a time bucket, such as the current date, as the partition key (PRIMARY KEY (date)).
• Consequence: Every write made today has the same partition key, so all current writes land on the nodes that own today's partition while the rest of the cluster idles.
• Note: A precise timestamp as the partition key would not do this (distinct values hash to different nodes), but it scatters each time range across the whole cluster, so you could not read a range by key.

2. Celebrity / high-volume entity hotspotting

• Mistake: Using tenant_id or influencer_user_id as the partition key when a few tenants or users are far bigger than the rest.
• Consequence: One very large tenant concentrates its load on one partition, which can hit that partition's limits while the others idle.

3. Write sharding (salting) the key

Spread one hot key's writes over $K$ partitions by adding a suffix. DynamoDB's guide gives two variants:

• Random suffix: e.g. 2026-11-13.1 to 2026-11-13.200, chosen at random per write. Writes spread evenly, but to read a day you must query every suffix and merge the results.
• Calculated suffix: derive the suffix from something you query by, such as a hash of the order ID:

Writes still spread evenly, and a single item can be read directly because its suffix can be recomputed; reading the whole day still needs all $K$ queries.

Polyglot persistence

Instead of one database for everything, many systems use different data stores for different kinds of data, choosing each for its access pattern:

Martin Fowler, who popularized the term, is clear about the costs: more complexity (each store has its own interface to learn and tune), new questions about consistency and availability, and no longer a single transactional source of truth.

Keeping the stores in sync: change data capture (CDC)

Writing to two stores from application code (a "dual write") is fragile: if the process crashes or the network fails between the two writes, the stores silently disagree. Change data capture instead reads the source database's own change log (MySQL's binlog, PostgreSQL's logical replication stream) and publishes each change, for example with Debezium into Kafka, for other stores to apply. Debezium delivers every change at least once: after a crash, consumers may see duplicates, so they must apply changes idempotently.