The sloppiest Friday prod-push in history
TL;DR: I finally have an email-update landing page for Malparse! Getting there this morning was a wake up call that I need to work way harder on building out a staging environment before I just push... and why everyone says not to push to prod on a Friday.
What Happened
Basically, I had my React front-end built out and ready to go. I had the back-end more or less finished, it was a simple one-endpoint Express app that would handle the user email input and the Mailchimp API interactions. I changed a couple of things around, created my Nginx config, ran npm run build and... errors.
I had an NPM versioning issue to solve, no big deal right?
I still have no idea how that lead to a server getting bricked and me losing my SSL private keys, but somehow, that happened...
I stood up a new server, changed the DNS A records, pushed the prod build code, generated a new SSL cert, updated the Nginx config again to handle the API endpoint requests... then updated the conf and the backend code again when it didn't work... and again... and again...
This repeated probably a dozen times. It was so terribly obvious that this was my first time trying to push a real project to prod, and very obvious that I didn't have CI/CD in place yet. That is the very next thing on the docket.
What we learned
- Back up your SSL cert
- DNS propagation takes way too long
- Have a solid staging environment that exactly mirrors your prod environment before you actually push to prod.
- Maybe actually think about CI/CD?
You can now, finally, sign up for updates, discounts and news from Malparse at the link below!
