DevSolved Logo DevSolved
Search... ⌘K
Write Log In Sign Up
Community Feed

Join DevSolved

Share war stories & investigate outages.

Sign Up for Free Log In to Account
ESC
↵ Search all
ESC Close
Pro tip: Type at least 2 characters

aws Incidents & Architecture Postmortems

Explore real-world production outages, root cause analyses, and solutions related to aws. Learn from the engineering community to prevent these bugs in your own systems.

3 Reports
0 Followers
Trending other
AWS Kinesis 2020: US-EAST-1 Cascading Failure
Avatar Devsolved • Nov 28
Resolved

AWS Kinesis 2020: US-EAST-1 Cascading Failure

DIAGNOSTIC TRACE
48h spent critical
Adding new capacity to the Kinesis front-end fleet pushed the servers over the operating system maximum thread limit, breaking internal communication and taking down Cognito, CloudWatch, and Lambda.
#aws
5930 0 120000
AWS S3 US-East-1 Outage: The Typo That Broke the Cloud
Avatar Devsolved • Jul 26
Resolved

AWS S3 US-East-1 Outage: The Typo That Broke the Cloud

DIAGNOSTIC TRACE
critical
An authorized S3 team member executed a command intending to remove a small number of billing servers. A typo in the command parameters removed a massive set of storage subsystem servers instead.
#aws
5120 0 110000
AWS S3 US-East-1 Outage: The Typo That Broke the Cloud
Avatar Devsolved • Jul 26
Resolved

AWS S3 US-East-1 Outage: The Typo That Broke the Cloud

DIAGNOSTIC TRACE
critical
An authorized S3 team member executed a command intending to remove a small number of billing servers. A typo in the command parameters removed a massive set of storage subsystem servers instead.
#aws
5120 0 110005