Featured Investigations
Critical production outages and high-impact fixes
BGP Route Leak: How a Small ISP Took Down Google
A tiny internet service provider in Nigeria accidentally advertised routes for Google's IP space. Major telecom networks blindly accepted these routes, blackholing traffic to Google services globally.
Devsolved
•
Jul 26
Knight Capital 2012: $440 Million Lost in 45 Minutes
A deployment of new trading software left a dead code path on one of eight servers. When activated, the server went rogue, buying high and selling low at algorithmic speed.
Devsolved
•
Jul 26
AWS S3 US-East-1 Outage: The Typo That Broke the Cloud
An authorized S3 team member executed a command intending to remove a small number of billing servers. A typo in the command parameters removed a massive set of storage subsystem servers instead.
Fastly Global Outage 2021: A Single Customer Regex Took Down the Internet
An undiscovered bug in the VCL (Varnish Configuration Language) compiler triggered a catastrophic global outage when a single customer updated their configuration with a specific regex pattern.
GitLab 2017: Primary Database Deleted via Accidental `rm -rf`
An engineer accidentally deleted the primary PostgreSQL database directory on the production server instead of the staging server, resulting in a 300GB data loss and an 18-hour outage.