Availability zones, failover and auto scaling

Regions and Availability Zones

AWS defines an Availability Zone as "one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region". The zones in a Region are many kilometers apart, all within 100 km of each other. They are joined by low-latency links that AWS describes as fast enough for synchronous replication between zones. A Region is a separate geographic area with its own set of zones.

A zone is a unit of failure. The primer's Users++ step spreads web servers and the database over several zones, so that losing one zone does not take the service down.

---

The load balancer and its health checks

An Application Load Balancer sends each request to a registered target. Cross-zone load balancing is always on at the load balancer level. It spreads requests over the targets in every enabled zone, and you can turn it off per target group.

The load balancer only stops sending to a target after health checks fail. The defaults for instance and IP targets:

• A check every 30 s, with a 5 s timeout.
• 2 consecutive failures mark a target unhealthy; 5 consecutive successes mark it healthy again.

So when a zone fails, its targets keep receiving their share of requests until interval × threshold has passed: 60 s with the defaults. If every target is unhealthy, the load balancer fails open and sends requests to all of them.

When Auto Scaling removes an instance, the load balancer first stops sending it new requests. It then waits for the deregistration delay (default 300 s) so that requests already in progress can finish.

---

Enough capacity to lose a zone

There are two ways to survive the loss of a zone:

1. Replace the lost capacity. Auto Scaling launches new instances in the surviving zones. This depends on the launch working during an incident, and requests are dropped until it completes.
2. Already have it. AWS calls this static stability: the system keeps working "without having to launch any new EC2 instances". With three zones, AWS overprovisions by 50 percent, so each zone normally runs at about 66 percent of the load it was tested for.

If $n$ instances carry the load and there are $z$ zones, each zone needs $\lceil n / (z-1) \rceil$ instances. For example, 30 instances in three zones becomes 15 per zone, 45 in total. Try both approaches in the visual.

---

Auto Scaling policies

The primer's Users++++ step scales on time of day or on CloudWatch metrics. Amazon EC2 Auto Scaling offers several policy types:

• Target tracking keeps a metric near a value, such as average CPU or requests per target. AWS recommends it when capacity should follow the metric in proportion.
• Step scaling adds or removes capacity in steps, depending on how far an alarm is breached.
• Simple scaling makes one adjustment per alarm, then waits for a cooldown (default 300 s) before the next. AWS advises against simple scaling and cooldowns. Target tracking and step scaling use instance warmup instead.
• Scheduled and predictive scaling add capacity ahead of time. Predictive scaling forecasts from daily and weekly patterns in past load. It suits slow-starting applications and recurring peaks like the primer's business-hours traffic.

Whatever the policy, new instances take time to launch and warm up. That is the delay the primer warns about.

---

Aurora: shared storage across three zones

In RDS for MySQL, each database instance has its own storage, and Multi-AZ keeps a standby copy in another zone. Amazon Aurora instead separates the database instances from a shared storage volume:

           [ Writer instance ]        [ Aurora Replica ]
                    \                     /
     +-------------------------------------------------------+
     |  Cluster volume: every 10 GB segment has 6 copies     |
     |  Zone a: copy 1, copy 2                               |
     |  Zone b: copy 3, copy 4                               |
     |  Zone c: copy 5, copy 6                               |
     |  Write quorum 4 of 6; read quorum 3 of 6              |
     +-------------------------------------------------------+

The Aurora paper (SIGMOD 2017) explains the numbers. With six votes, a write quorum of 4 and a read quorum of 3:

• Losing a whole zone (2 copies) still leaves 4 copies, so writes continue.
• Losing a zone and one more node leaves 3 copies, enough to read and to rebuild the missing copies, though not enough to write.

Only redo log records cross the network. In normal running the database reads from a single up-to-date segment; quorum reads are needed only during recovery.

Failover. If the writer fails and an Aurora Replica exists, Aurora promotes it. AWS says this "typically" takes "less than 60 seconds, and often less than 30 seconds", and the cluster endpoint always points to the current writer. Without a replica, Aurora creates a new writer, typically in under 10 minutes. The paper describes "failovers to replicas without loss of data": the replica reads the same storage volume as the writer.