New Video Infrastructure
by Nick FogleOur old duct-taped backend got us to over 1,000 customers before we hit scaling issues: servers started running out of memory, the job queue would get backed up, and customers were waiting longer and longer for videos to finish. This was something we put off for as long as we could.
We spent two months rewriting our entire backend and now we're really starting to see the benefits. Our AWS bill decreased by more than 50%, video wait times dropped by several minutes, and the system is much more robust with increased logging visibility and retry logic.
-
2
Good work
-
1
Really cool. Do you have more info on how you dropped your aws bill by 50%?
-
1
Sure, happy to give some context. It's mostly the difference between relying on EC2 for video generation vs. AWS Lambda.
Prior to the update, we were paying for a bunch of C4.xlarge instances handling the video processing load. When wait times started getting out of hand, I'd just spin up a new one and throw it behind our video worker. I'd experimented with quite a few instance types, and the C4s were the best bang for our buck. Eventually we were getting diminishing returns from each additional instance, and we'd had our eye on Lambda for a while.
It was a massive effort to get all of our video infrastructure rewritten, but after a few months we got it done. Looking back, I don't think we'd be where we are now if we hadn't made the switch.
-
1
hey , can you please elaborate , how did you used lambda to reduce the price?
what are using lambda for ?
the video processing ?
Thanks allot -
1
Wow didn't know AWS Lambda made such a huge difference. Did you also investigate Google Cloud Services?
-
1
Not really. We had a bunch of other services on AWS, so it just seemed easier to stay put when building out the new architecture. I think prices are roughly the same between AWS, Azure, and GC's serverless functions, so it's probably more a choice of where your team is more comfortable.
-
1
Thanks. It's just me for now. :)
-
-
-
-
