AWS Reports Increased API Error Rates in Asia Pacific Region

This morning, Amazon Web Services reported "increased API error rates" in the Asia Pacific (Sydney) Region. The first reported errors affected only Amazon Elastic Compute Cloud (EC2). As AWS looked into the issue, it later found that the errors also impacted Appstream 2.0, Elastic Load Balancing (ELB), ElastiCache, Relational Database Service (RDS), Workspaces, and Lambda.

A few hours after the issue was first discovered, AWS reported that it identified the root cause and was working towards a resolution. As the investigation progressed, AWS identified as many as seven services affected by the errors but reported that the errors were isolated to AP-SOUTHEAST-2 Region for functions configured with VPC settings. Once the company determined the root cause, it explained:

"A data store used by a subsystem responsible for the configuration of Virtual Private Cloud (VPC) networks is currently offline and the engineering team is working to restore it. While the investigation into the issue was started immediately, it took us longer to understand the full extent of the issue and determine a path to recovery."

Eventually, AWS determined that it needed to restore the data store to the point before the issue began. It did so, which required the company to disable writes. That process took a few hours; and upon completion, the company apologized for the impact and the issue is now marked as resolved on the AWS service health dashboard.

Be sure to read the next Cloud article: Google Stackdriver Cloud Monitoring Dashboards API Now Generally Available