Performance versus scalability

The fundamental distinction

A system has a performance problem if it is slow for a single user under zero load. Common causes include unindexed database queries, synchronous blocking I/O, heavy cryptographic hashing on the request path, or unoptimized rendering routines.

A system has a scalability problem if it is fast for one user, but degrades or fails as traffic increases. Adding concurrent users causes thread pool exhaustion, database connection saturation, lock contention, or network bandwidth depletion.

Werner Vogels defines a service as scalable if adding resources increases its performance in proportion to the resources added. Ideally, doubling the servers doubles the throughput:

Vertical scaling (scaling up)

Scaling vertically means adding more CPU cores, RAM, and faster NVMe storage to a single server instance.

• Advantages: Zero architectural changes; maintains strict ACID transactions without distributed consensus; simple single-node debugging.
• Disadvantages: Hard hardware ceiling; diminishing returns due to bus and memory contention; the largest machines cost disproportionately more; one machine remains a single point of failure (SPOF).

Horizontal scaling (scaling out)

Scaling horizontally means adding more discrete compute nodes and distributing requests across them via load balancing. Scaling out on commodity machines is usually more cost-efficient and more available than scaling up one expensive server, but it adds complexity.

• Stateless compute: To scale web servers horizontally, session state must be removed from local server memory and kept in a shared store (a database, or a cache such as Redis) or in signed client-side tokens (JWTs).
• Downstream load: As the web tier scales out, caches and databases behind it must handle more simultaneous connections.
• The load balancer itself: It can become a bottleneck, and a single load balancer is a single point of failure, so production systems run more than one.