What a great campfire story Casey stumbled into. Whether any of this is true few will ever know, but its very reasonable that a race condition and a stalled job to apply configuration caused by someone who left the company 10 years ago caused an outage. I find it hilarious that they call this guy he answers, yup I still know the password, but how do I know you’re legit, I’m not just handing out the password. Casey did a stand up job telling this story.
Posts tagged: cloud
All posts with the tag "cloud"
7 posts
latest post 2025-12-06
Publishing rhythm
Another Big Cloud Outage Nov 2025
Today I woke up to finding out that cloudflare hade a widespread outage. My Reader uses tailwind cdn for styles and it was down. Otherwise it was not so impactful to me and felt kike they were quick to have it up.
I’m not really researching here, just jotting thoughts down from a parking lot waiting for pickup.
It feels like we are seeing a lot of these lately. They feel much more frequent. It feels like a whole industry was sold on 9’s and reliability of big cloud that we just aren’t getting.
...
Cloud is cooked bois. Seriously too much dumb shit relies on the cloud. Too much critical shit relies on single AZ’s. If normies are literally loosing sleep over an AWS outage (queue the Uncle Roger Voice), You’ve Fucked up. It’s wild to even think about a bed relying on the cloud let alone fully stop working when UE-1 goes down. I want to live in a world of opt in FEATURES, things that bring value to a product because it makes it better. Somehow a bed smells suspiciously like a cash grab for a subscription because its cloud connected. And yet for some reason it takes 16GeeeBee’s per month. I don’t own one of these, and I don’t want to. I don’t want a subscription for everything, I want my shit to just work. The future we are headed towards a world that is ever more reliant on a few key clouds. Which is fine. It’s fantastic that small companies can start and scale without owning an infrastructure team. It’s great that they have the ability to give us many nines of reliability. Some things just don’t need the cloud.
This is a really great animation of latency across different aws regions based on us-east-1.
This episode really got me thinking about the difference between HA and DR and my approach to each one. They talk about it from the perspective of a cach cow kind of app rather than a homelab or internal tooling, but think of HA as 9’s how many 9s are we willing to pay for, tink of DR as dollars how many dollars will we loose during the period of recovery. So much more in the episode, a lot of talk around cloud vendors and what they give you vs a purpose build platform with HA and DR in mind.
switching cloud providers, theres no easy way. K8s was supposed to get us there, haha, the deep integrations with each vendor just keep locking us in
Super useful way to show a tree view of an s3 bucket’s structure!