So with the Cloudflare 502 outage for 20min just now, both Trennd and my other indie side project went down. (The other serves 14 paid B2B customers that I fought tooth and nail to win and would hate to lose over something like this!)
With their otherwise awesome service and generous free plan, I bet a lot of other indiehackers here rely on Cloudflare...
So I was wondering who else was affected?! And how we could mitigate the risk to our IH products in the future... especially against relying on single point of failure systems like this in general?
Saw more intermittent slow downs more than anything else. I monitor a bunch of entry points into my service (as one should) and not all of them were affected. While not ideal, it could have been worse.
Our up time is still over 99.9% :)
Not sure how to mitigate SPOF at the DNS level. I actually didn't see any errors on pages that were using CF protection, so it's not like it was isolated to protected pages (which could have been mitigated with a quick switch to turn off protection).
Not to say you shouldn't worry about losing your customers, but unless you've given them an SLA and they are holding you to it, it's probably fine. Nothing wrong with letting them know about the issue, what was in and out of your control, and how you are planning to address things in the future.
I've fucked up a few things in the past and have been up front with my customers and have always been met with a ton of empathy, especially from fellow developers that know the struggles.
Yeah, I don't have an SLA and it was thankfully only 20min or so. I think you're right though about just being open with your customers and getting ahead of it by talking to them.
I've sent them a quick email noting and explaining the issue in case they noticed any downtime. The one that replied so far was pretty cool about it!
PanelJam.com went down...
Along with our blog website, online shop, and ComicsForBlogs.com
On top of that, I noticed Notion.so was broken, and Discord too
Notion and discord too?! About 10% of the internet relies on Cloudflare so I guess I shouldn't be so suprised!
Love the PanelJam design and fonts BTW!