Skip to content

Designing for Resiliency

Bill Nelson Dec 11, 2021 12:18:31 PM

On Tues, Dec 7th, Amazon suffered one of the largest outages in recent years as both businesses and consumers were impacted for more than 5 hours.  Lines were backed up at the Magic Kingdom as park goers had to be verified manually and streaming services such as Netflix and Disney+ went dark.  Alexa based home automation systems ignored commands from their masters and some people couldn't even get into their own homes.  Even Amazon, itself, was affected as its own massive e-commerce operation was unable to take orders.  

What I find interesting is that the problem was primarily localized in the US-EAST-1 region and yes, this is their primary region, but what about high availability and failover to other regions?  Does this recent outage indicate that applications and services for major companies such as Netflix, Disney, and Delta Airlines are localized to the US-EAST-1 region?  I doubt it but it does make me wonder.

Two weeks before the outage, Identity Fusion completed an AWS multi-region identity management deployment where users are routed to the region nearest to them. European and US East coast users were sent to US-EAST-1 and Asian and US West coast users were sent to US-WEST-2.  How many of our client's East coast customers were affected by the AWS outage?  Not a single one as East coast traffic was automatically redirected to the AWS West region and business continuity was completely unaffected.

Did we monitor the situation as it unfolded? Certainly. Were we worried that we may have missed something? Absolutely. But as the day went on and the architecture was proven to work as designed, our concerns lessoned. This does make me wonder, however, why our customers were not affected and so many big named Fortune 100 companies were?

 

Select the Contact Us button at the top of this page if you would like to learn more about how we can build a highly available / scalable / performant identity and access management solution for your company. 

 

Leave a Comment