Do you backup your SaaS account data?
If like I do, you have projects that are very dependent on the data held in a array of SaaS apps (Slack, AirTable, Trello, Firebase etc), do you ever think of backing that data up in case you lose access to your account? If so how?
Yes, though as to where and how it depends!
I like to keep my backups in the same one or two places. That way my security and data retrieval is simplified. By default I use AWS. For databases hosted there, I use built in snapshotting and retention tools. Heck, that functionality alone makes using RDS worth the price. Snapshotting Lightsail and EC2 instances isn't a bad idea either.
For everything else, I use S3 and ensure that my buckets are well secured. In some cases I manually move backup files there, in others I've set up some automation via my own scripts (mostly Python and Bash) to grab data and dump it to S3. A bit of work to set up, but reassuring to know it's running in the background.
The exception for me is anything that's already in Google's ecosystem. There I use Drive and again ensure that my backup folders are properly secured. There's no reason to move it from Google->AWS. One less process I need to keep an eye on.
In the end, I figure storage is incredibly cheap. I'd rather back up more than I need. The downside is another place to worry about data security, so I find that to be the most time consuming aspect to set up and test.
"Snapshotting Lightsail and EC2 instances isn't a bad idea either" I'm not too familiar with AWS, but am I right in thinking that these are essentially VM based services rather than Container based? Because backing up VM's makes sense where they can be stateful, rather than containers that usually aren't.
That’s correct. I often snapshot after an initial deploy or some other key config change. It’s saved me at least once.
That is a great question, and it deserves two answers.
The short answer is "Yes". Having lost a lot of data over my career due to services shutting down or losing my data in an incident, I have developed a habit of regularly backing up data from services that allow that. Where possible, I use automation (scripts for Apple Scripts things like Evernote full export, or bash scripts for API-based services that crawl and download all data)
The long answer is "Yes, and more".
Any service that will not allow me to export my data in some form is immediately considered a business risk. And that can be okay, for certain things like nice-to-have analytics, but will mean I will never use that service for business-critical data.
People will tell you to self-host your data, run your own database and build your own backup system. I recommend doing that, and then STILL using professional backup tools for those services. AWS S3 is a popular target for data backups, and I recommend eating that cost to make sure that you have some form of data snapshot for your business-critical data there.
When you pick a service to host your database, make sure they offer easily retrievable and restorable automatic backups. Don't save money here. One crash, one faulty script, one new hire that deletes the prod database instead of their local copy, and your business will suffer lethal damage.
For any service that you use that does not offer this option, make it a habit (or a delegated task for someone) to do a manual backup at least every month. Use Mailchimp? Export the email lists. Intercom? Get the customer lists and export conversations. Slack? You can get a full export from the admin interface there.
Doing this once a month will give you a number of interesting insights into your service usage, data structure and may surface growth metrics you have not yet thought of.
Thanks for the thoughtful response!
"Any service that will not allow me to export my data in some form is immediately considered a business risk." When have you faced this decision in the past?
"Doing this once a month will give you a number of interesting insights into your service usage, data structure and may surface growth metrics you have not yet thought of." Pretty neat, I never really thought of backup data as somewhere you can get additional insights from, but it makes sense. Perhaps if you uploaded it all to something like BigQuery that lets you query structured files. Any examples of this too?
Business Risk:
Any kind of data processing system that would not be able to export your data as a CSV or at least XLS. There are some great tools our there like AirTable and Notion, that will allow you easy exports of all your data. And then there are some that don't. Those I avoid (to name and to use). Any research into a new business tool should immediately include data export. If they need you to ask support to do it for you, that's also fine, as long as you have the right to get all of your data.
Backup Data Introspection:
I can't give you specific examples, but let's say you have two backups of your database, which include some non-personal customer data that you're allowed to analyze. You could look into things like: between backup A and backup B, how many new records were added? How many old records where updated? What's the percentage of new additions to updates? If those records are text-based (because your product is for people who write a lot), did the average word count change for that user? Over all users? Do users who have higher word counts tend to add more records or change old ones?
Taking deltas of snapshots and then comparing the deltas can be very insightful. You can then build features that extract/track that data and offer it to your customers as per-customer statistics, or (like what Baremetrics is doing so wonderfully with their Benchmarks feature) cluster them into cohorts and show them where they stand. For our example, that would mean showing them stats like "You are in the cohort of writers who write between 3 and 5 articles a week. You're outranking 65% of the writers in your cohort when it comes to the simplicity index of your writing". Stuff like that.
When it comes to business information, that is the same thing. You look into your Uptime Monitoring service export, feed that into a big database and then see the month over month trends. How many alerts did we get? Are there latency metrics we could average and see a trend in.
It's like you can be a little internal data scientist :)
always backup your data.
there are:
database data: from automated cloud provided to bash scripts, prefer self managed dbs, scripted backups have their own pitfalls
image cluster data: k8s, droplets, ebs
account data: api keys passwords etc. this includes master db password aws key/secret, DO token ..
for account data you can use a AWS's secrets manager,
my own pkhub.io :) or at a minimun use a password manager.
Like you product - must of taken a lot of reading / experience to be able to build something like that securely. Is security your background?
thanks, it took a ton of work and research. Yup, my background is software engineering and security. Still improving e.g this week I added dnssec. Last four weeks I ran two separate bounty rounds with two security researchers, they only found minor things in http headers and rate limiting, so security is pretty tight.
Also getting the offline storage secure took time and research, but basically the whole db can be exposed and the data will still be safe. Data is encrypted using aes cbc 256 and hmac512 with stretched/derived keys from your master password and peppered with our own private key.
Yo, I saw that you were in SUS 2019. I am as well. I wanted to invite you to my slack group for Open-source MArketing and getting unstuck in terms of growth and ideas if you were having trouble with marketing or were curious.
https://bit.ly/2Mq0D6Q
Most of my important customer data is in one place. In a MongoDB database which is backed up hourly, daily, weekly, and monthly with differing retention policies for each interval (from hourly kept for one day to monthly kept forever). The database is on GCP and the backups are on both GCP and AWS S3. (Yeah, it's probably overkill)
Everything else that is in SaaS apps, I don't worry about too much. I pay for SaaS so that they can worry about that instead of me.
Nah, implementing DB backups that are copied across multiple cloud providers isn't overkill, I'm going to need to implement something similar for my SQL data :) It only took reading one horror story on HN about someone losing their GCP login over an automated fraud prevention algorithm gone wrong to convinced me of that!
My main concern here is data stored in SaaS platforms that you don't program yourself - Slack, AirTable, etc. I can't decide whether it's worth it to implement automated procedures not unlike database backups or not.
I more meant the variety of backup frequencies being slightly overkill. Although, if I lost access to my Google login, I'd have bigger problems to worry about.
Yeah, if it were easily automatable, I'd do it. But that sort of thing tends to not be very easy. Plus, half the point of paying for SaaS over self-hosted tools is that you don't have to worry about backups and managing data.
I keep my data in my own db, which has daily and weekly backups.
Daily backups are images of my entire DO droplet and db. Weekly backups are done through a bash script that gets a db dump and moves it with scp.
What do you get from imaging your droplet as well as the db?
This comment was deleted 7 years ago