I will be referencing a few tools without much elaboration. If you have any questions about them, please ask!
This is more of a cautionary tale than anything else. The lessons are obvious in hindsight: “use what you know” and “be careful of new technology”. What I want to do is explain how I fell into making these mistakes so others can avoid them.
So, I wanted the infrastructure for my night-and-weekends project to be reasonably lean in terms of cost and complexity. For reference, I use a lot of AWS at my day job and it’s not cheap. I wanted to explore other cloud providers and had the following wish list:
DigitalOcean caught my attention for a few reasons. First, it is half the cost of AWS. Droplets are cheap and provide remarkably good compute value. Second, they fulfill all the technical requirements above and are actually the only small cloud provider that offer managed databases as a service. Third, as an indie developer it feels more indie-like to use a smaller provider.
The documentation was straight forward and I was able to get a DigitalOcean fully environment running in a handful of evenings. Most of the time was actually spent setting up the image with Packer. It is not soon after that I started having problems.
I ended up having significant issues with DigitalOcean’s database and load balancer services. Droplets had 50% chance of not automatically being given network access to the database which meant I needed to do it manually each time. Worse yet, the load balancer started to give me more and more 503s as time went on as well. My guess was it was not properly removing droplets after they were being deleted (I was constantly creating and deleting droplets as part of my releases). Plus, there were other bugs too! I didn’t want to spend my precious time debugging all of this with their support.
As much as I want to blame everything on DigitalOcean, it was my fault as well. I am skeptical of using a brand new library in my code and I did not have that same caution when it came to my infrastructure. Their database service became “generally available” less than a year ago! To their credit, DigitalOcean has been fairly aggressive in the past couple of years to add and improve their non-droplet services. Maybe in a year DigitalOcean’s newer services will become more reliable, but I don’t have a year to wait.
Back to AWS’ well-monied arms I went.
Getting everything running again on AWS cost me a weekend. I was already familiar with it and nothing about my DigitalOcean infrastructure was proprietary. For example, all I needed to do on Packer is change the build target from DigitalOcean to AWS and a few small script changes. Using non-proprietary technology (e.g. Terraform, Packer) gives you an exit strategy for free. Generally having an exit or alternative strategy whenever you are exploring something unknown is a good idea.
Using technology you already know (in this case, AWS) is a huge time saver and even though DigitalOcean is cheaper, at my current stage of my life my time is more expensive.
Admittedly, I think what made me truly stray away from AWS was the thrill of learning something new. The dark truth was that I didn’t really have to use DigitalOcean. It is pretty to find yourself with a few thousand dollars of AWS credits for a couple of years if you know where to look. Plus, it’s still possible to learn new things with technology you already know! When I migrated to AWS I decided to try setting up an ALB for the first time and bam, I learned something new in a smaller, safer way!
In hindsight, I should have started on AWS and only transitioned to DigitalOcean once cost became an issue. It was a classic case of premature optimization.