Post-Mortem

9 posts

A new DDoS rule resulted in an increase in error responses and latency for Cloudflare customers. Here’s how it went wrong, and what we’ve learned

Lloyd Wallis6/26/2024

Just four months after a complete power outage at a critical data center we were hit with the exact same scenario. Here’s how we did this time, and what’s next

Matthew Prince4/8/2024

Beginning on Thursday, November 2, 2023, at 11:43 UTC Cloudflare's control plane and analytics services experienced an outage. Here are the details

Matthew Prince http://blog.cloudflare.com/author/matthew-prince/11/4/2023

Multiple Cloudflare services were unavailable for 37 minutes on October 30, 2023, due to the misconfiguration of a deployment tool used by Workers KV.

Matt Silverlock http://blog.cloudflare.com/author/silverlock/11/1/2023

On Wednesday, October 18, 2023, we discovered attacks on our system that we were able to trace back to Okta. We have verified that no Cloudflare customer information or systems were impacted by this event because of our rapid response.

Sourov Zaman http://blog.cloudflare.com/author/sourov/10/20/2023

On 4 October 2023, Cloudflare experienced DNS resolution problems. Some users may have received SERVFAIL DNS responses to valid queries. In this blog, we’re going to talk about what the failure was, why it occurred, and what we’re doing to make sure this doesn’t happen again

Ólafur Guðmundsson http://blog.cloudflare.com/author/olafur-gudmundsson/10/4/2023

On 4 October 2023, Cloudflare experienced DNS resolution problems. Some users may have received SERVFAIL DNS responses to valid queries. In this blog, we’re going to talk about what the failure was, why it occurred, and what we’re doing to make sure this doesn’t happen again

Ólafur Guðmundsson10/4/2023

A deep dive into the recent incidents relating to Workers KV, and how we’re going to fix them

Matt Silverlock8/2/2023

On July 9, 2023, users in the Asia Pacific region experienced connection errors due to origin DNS resolution failures to .com and .net TLD nameservers

Christian Elmerot7/11/2023