Home
Starting Up
Case Studies DB
Products
Ideas DB
Vibe Coding Tools
Subscribe to IH+
Starting Up
Case Studies
Ideas DB
Products DB
Join
15
Likes
18
Comments
Atlassian down for 12 days and counting 😬 Can you imagine?
by
Jacob H
https://newsletter.pragmaticengineer.com/p/scoop-atlassian/
I remember Indie Hackers going down for 12 hours once in the early days. A story on the site went viral, and I was out of the house and not checking my phone.
I was mortified when I found out. This was back before I was making enough money from the site to pay my bills, so I needed every win I could get. Going viral was rare, and I had missed out on all of the traffic.
But I realized there was a silver lining. Besides me, nobody really cared that Indie Hackers was down. It isn't a mission critical site. People's lives and businesses don't depend on it. I don't think I'll ever start a website or business in the future where that isn't the case.
One time, during YC- a couple days before demo day I deleted a significant portion of the production databases data.. ~50 companies profile data, gone.
I remember feeling like I just got punched in the gut. Terrible feeling.
We managed to resolve it with a mixture of logs, a db backup and some hard conversations.
I still get a hard time about it from my cofounders. 😂
These things feel like the end of the world when they happen, but eventually they just turn into cute stories from the good ol' days.
The flip side: mission critical products provide more value to the world (almost by definition)
Well, at least people waste less time on being bureaucratic on their Jira tickets...
Great post @jacobh. Excellent breakdown!
You’re right, this type of outage is unheard of for a company like Atlassian who’s primary customer needs are availability.
Much like AWS, this type of outage highlights the dependence we all have on so few. It just gives me the absolute fear🤣
I guess this is just another reason other companies like ClickUp are entering the space. Trello is also going to see probably an initial spike from this. JIRA has apparently 84% market share, so I guess people are beginning to the see reasons why its bad to only depend on JIRA and not to give other competitors a shot.
Still, the outage seems somewhat minor, impacting only 400,000 total employees/users. This isn't small, but it's not like millions of customers can't use their product.
Any type of delay like this is awful, but at least its not a cyberattack.
This comment was deleted 4 years ago
That's unacceptable, and I would take my business elsewhere in a heartbeat no matter how much effort it took.
12 days is no longer a technological failure, it's an organizational and business failure.
We should all be running a database with one lagging replica (one that's applying changes xx hours after they happen)...
The business I work my day job at (Rewind) does backups on all kinds of services like this. I think in terms of Atlassian products we just do Trello and Jira for now, but working on many more
That's why i wasn't able to create an account on Trello??
We’ll (the community) will probably get amazing content from confluence once they’ve put out the fire and their engineering team can breathe.
I’m looking forward to blog posts written by engineers working there describing their experience and how they dealt with the situation.
In a few months we’ll probably also get some blog posts about things they’ve innovated on to mitigate risk of similar events happening.
Terrible event for the company, but it will push the needle for the dev community.
Murphy's Law at its best!
My business partner pinged me on Thursday to say he was demoing our app to some international clients and it was busted. Ended up being a commented line of javascript.
Was annoyed with myself but don't feel as bad now reading the root cause explanation of what Atlassian did to break their platform
https://www.bleepingcomputer.com/news/technology/atlassian-finally-explains-the-cause-of-ongoing-cloud-outage/
Heroku disabled GitHub integration.
It was makes me feel a bit better when the big companies have outages, just goes to prove its unavoidable no matter how hard you try, but the case of Jira being down for so long is crazy - don't recall an outage ever lasting as long for another company. I wonder what the impact, if any there is is on their business moving forward for existing customers
Man, I'd be freakin out if that was my product.
Side note: Glad I don't use Jira. Post-it-note kanbans for the win!
Yeah I've always used good, old-fashioned pen and paper for my projects
One of my clients uses Monday and that's been down a lot this week too, so that's weird.
This comment was deleted 4 years ago