Cloud Computing Cost Analysis

👋 Hackers
I recently dove into calculating cloud costs for my next project, and I thought it would be good to share my initial results. This is based on a simulated early access program, so there are some guard rails. The total costs shown are based on a 30 day month. I share a link to some of the resources I gathered in a Notion notebook at the end.

Background: my application would be cloud-heavy and rely on Kubernetes fundamentals to ensure high availability for a service that is usually difficult for developers to get up and running: AI inference. The basic idea is that training an AI model is somewhat straightforward, but putting it on a server and using it on the internet is not so. So this would be like Heroku but for AI models.
(www.EasyTensor.com 👈👈👈 If you'd like to join the early access list)

💽 Application storage:

• all models are stored in object storage
• deployed models are stored on NFS for fast access
• deployed models are loaded to memory

🦜 Early Access Assumptions:
Users: 100
Model to store (disk) per user: 20
Model to deploy (RAM) per user: 5
Max size of model (GB): 0.50
Total object store needed: 2,000 GB
Total disk needed: 250 GB
Total memory needed: 250 GB

☁️ Cloud Providers Compared:
AWS (Amazon) and GCP (Google)

🖥 Compute Cost:
Since this application is memory intensive, I will be bound by how much RAM i can have per instance, rather than vCPU. Comparing AWS and GCP offerings, the most efficient instance types ($/GB/hour) are e2-highmem-4 (0.005649375) and r5.xlarge (0.007875).
GCP: 0.005649375 *250 * 24 * 30 = $1,016 (GCP wins)
AWS: 0.007875 *250 * 24 * 30 = $1,417

💽 Network Storage Cost: since this is a kubernetes application, we assume the storage will be shared across a Network File System (shared among servers). Both GCP and AWS have the same price for the standard storage, which is $0.30 GB/month.
0.30 * 250 = $75

🗑 Object Storage: we will store all the models as objects in an object-store like S3 or Google's offering, since it's much cheaper than keeping it on disk.
AWS: .023 * 2,000 = 46
GCP: .02 * 2000 = 40 (GCP wins)

Total GCP: 1,016 + 75 + 40 = 1,131 ✅
Total AWS: 1,417 + 75 + 46 = 1,538

😴Conclusion
GCP was about 25% cheaper than AWS for my use case, with the majority of the saving coming from memory-based savings on the servers. There are many levers to turn up and down based on your needs, so make sure to analyze what component of your application will need the most scale.

📝 Here is a notion notebook that has a few more details and other comparable prices for instance types
https://www.notion.so/Cloud-Cost-Analysis-for-Machine-Learning-9ce8d376b31e495ba114948d5455794c

P.S. Here are a few costs I decided to exclude from this comparison:

• Kubernetes upkeep cost (negligible)
• Local block storage (application doesn't need it)