Head to Head: AWS vs ADTAQ
So at ADTAQ VPS we sell Virtual Private Servers. Virtual private servers were the precursor to "the cloud". They are virtualized instances running on hypervisors in datacenters that are available for purchase by businesses and consumers, just like AWS.
...or are they?
Is what we sell as good as what AWS sells? Do we have any advantage whatsoever going up against such a behemoth in the cloud space? Should I get back to selling pet rocks on Craigslist?
Today, I set out to answer these questions and more; join me on a journey of product validation!
The Contestants
In this corner, wearing the green trunks with the creepy smile on them, weighing in at $82 BILLION dollars in annual revenue, the heavyweight champion of the worldwideweb, Amazon Web Services!
And in this corner, the challenger, from parts unknown, wearing the white trunks with letter A logo that is really just a free font found on the internet, weighing in at "almost enough to get a new coffee machine", ADTAQ VPS!
Ground Rules
Each Contestant will deploy a "storage instance" meeting the following criteria:
- 2 Virtual CPUs
- 2 GB RAM
- 2 TB Disk Space
- 2.5 Gbit Uplink
They will then be judged on two categories:
- Performance Metrics
- Pricing / Value
Performance Metrics will be measured using the open source Yet Another Benchmark Script available here.
Pricing / Value will be measured via the AWS Calculator and the far easier to comprehend transparent pricing shown on ADTAQ's Storage VPS product page.
Once the data from these categories has been captured, it will be up to YOU, the Indiehacker community, to choose a winner.
LET'S GET READY TO RUMB..er, copy+paste a bunch of stuff...
Performance Metrics
AWS: The Legend Continues
Browsing through AWS's EC2 Management Console, I attempted to find the closest thing to what we at ADTAQ sell as 'STORAGE 2'. In order to get "Up to 5 Gigabit" of network performance to match ADTAQ's 2.5 Gbit pipe, I needed to scroll down to the t3.small.
I then configured my networking and SSH key then kept scrolling until the Configure Storage section. It looks like a t3.small includes just 8 GB of storage, so I increased this to 2048 GB or 2 TB of the default gp3 storage.
I launched my instance and after a minute or so was able to log in.
I wasted no time (these are billed by the hour, you know) and fired up YABS. I'm going to break up each of the benchmark results one at a time.
fio Disk Speed Tests (Mixed R/W 50/50)
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 6.19 MB/s (1.5k) | 67.56 MB/s (1.0k)
Write | 6.20 MB/s (1.5k) | 68.02 MB/s (1.0k)
Total | 12.39 MB/s (3.0k) | 135.58 MB/s (2.1k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 65.17 MB/s (127) | 64.62 MB/s (63)
Write | 68.61 MB/s (134) | 69.35 MB/s (67)
Total | 133.78 MB/s (261) | 133.98 MB/s (130)
Alright, so what does this even mean? Well, IOPS is the acronym for Input/Output Operations per Second. Here's a nice table from Wikipedia:
Hmm, well this AWS instance seems to do very well compared to a SAS drive with a lower block sizes, but then seems to under-perform once block sizes increase to 512k.
Next up are the network speed tests. YABS uses iperf so disk speed won't act as a bottleneck, and it runs 8 threads in each direction in an attempt to really max things out. Here's how our t3.small did:
iperf3 Network Speed Tests (IPv4):
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 1.11 Gbits/sec | 944 Mbits/sec | 76.2 ms
Scaleway | Paris, FR (10G) | 1.05 Gbits/sec | busy | 78.4 ms
NovoServe | North Holland, NL (40G) | 1.29 Gbits/sec | 1.29 Gbits/sec | 84.2 ms
Uztelecom | Tashkent, UZ (10G) | 1.10 Gbits/sec | 321 Mbits/sec | 164 ms
Clouvider | NYC, NY, US (10G) | busy | 397 Mbits/sec | 7.17 ms
Clouvider | Dallas, TX, US (10G) | 1.93 Gbits/sec | 1.94 Gbits/sec | 30.0 ms
Clouvider | Los Angeles, CA, US (10G) | 1.09 Gbits/sec | 825 Mbits/sec | 56.2 ms
Cool, not too shabby really. As you may recall, the t3.small is categorized as "Up to 5 Gigabit" network wise so this seems reasonable.
Now, on to the challenger...
How does ADTAQ stack up?
Well, first I need to order an equivalent instance, so I will head over to https://www.adtaq.com/storage-vps/ and choose STORAGE 2:
After I click, it takes me straight to a Stripe powered Checkout page:
Give me a second to fill this out...
... ...
...
Okay CVC code, yeah....
Okay, and Subscribe
And now, just like that, I've got an email with a login to the ADTAQ VPS Control Panel.
Here's a snapshot of it. You can manage power on your system, check usage graphs, do re-installs, and manage reverse DNS from here. Oh, and you can activate a VNC virtual console too which is nice when you screw up the firewall rules like I totally never have done, ever.
Alright, time to run the same YABS benchmarks as we did on AWS:
fio Disk Speed Tests (Mixed R/W 50/50)
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 130.16 MB/s (32.5k) | 1.85 GB/s (28.9k)
Write | 130.51 MB/s (32.6k) | 1.86 GB/s (29.0k)
Total | 260.67 MB/s (65.1k) | 3.71 GB/s (57.9k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 7.85 GB/s (15.3k) | 690.09 MB/s (673)
Write | 8.27 GB/s (16.1k) | 736.05 MB/s (718)
Total | 16.13 MB/s (31.5k) | 1.42 GB/s (1.3k)
Okay, so... Wait a minute. Is this right? ADTAQ is doing 65,100 IOPS at 260.67 MB/s while AWS is doing 3,000 IOPS at 12.39 MB/s? Our disk is actually 21 times faster than what AWS is selling?
Yep, and here's why. All of the hyperscale cloud providers artificially throttle the IOPS on their instances. The highest any of the tests on the AWS system got was 1500 IOPS so I'm guessing that must be the default throttle value.
I'll be honest, even though I helped design our platform, I had no idea it would perform this well, and also had no idea just how severely the big cloud providers were throttling IOPS by default on their platforms. Can you pay more for more IOPS on AWS? Yes, you can... More on that later.
Now, let's test the ADTAQ Network. AWS is a global entity so I do not really expect to keep up with them here, but let's find out:
iperf3 Network Speed Tests (IPv4):
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 1.25 Gbits/sec | 688 Mbits/sec | 138 ms
Scaleway | Paris, FR (10G) | 989 Mbits/sec | 809 Mbits/sec | 149 ms
NovoServe | North Holland, NL (40G) | 1.16 Gbits/sec | 1.15 Gbits/sec | 145 ms
Uztelecom | Tashkent, UZ (10G) | 895 Mbits/sec | 755 Mbits/sec | 224 ms
Clouvider | NYC, NY, US (10G) | 1.92 Gbits/sec | 368 Mbits/sec | 65.2 ms
Clouvider | Dallas, TX, US (10G) | 2.22 Gbits/sec | 1.8 Gbits/sec | 63.3 ms
Clouvider | Los Angeles, CA, US (10G) | 2.63 Gbits/sec | 2.01 Gbits/sec | 37.1 ms
Alright, well, considering the AWS instance was in Ashburn (East Coast USA) and ADTAQ is based out of Quincy, Washington (West Coast USA) I'd say these numbers are pretty good. AWS pings lower to everywhere except Los Angeles which makes sense based on the location of the speed test data centers.
That said, the transfer speeds seem pretty equal. If the lowest possible latency to Europe and the East Coast was critical for your app, maybe the AWS us-east instance would be better. None of these speed tests head to Asia, and LA is the lone west coast test, but it's probably fair to assume that ADTAQ would win out in any tests on that side of the planet.
Pricing and Value
Well, in my opinion, the Performance section has established the following:
- ADTAQ's disks are far, far faster than a default AWS instance
- The networks are roughly equal, depending on which part of the world you are trying to reach.
So, what about price? It's already been established that ADTAQ's Storage 2 VPS is $20/month.
I am going to head over to https://calculator.aws now and key everything in to determine the equivalent price at AWS; feel free to play along at home. I started with this:
t3.small Amazon EC2 On-Demand instances: $15.18/month
2 TB Amazon Elastic Block Store (EBS): $160/month
TOTAL: $175.18/month
Okay so that's $155.18/mo more than ADTAQ's Storage 2 VPS. But wait, this isn't apples to apples. The AWS system was capped at 1,500 IOPS and 70MB/s. We need to increase those caps in order to make the comparison accurate:
t3.small Amazon EC2 On-Demand instances: $15.18/month
2 TB + 16,000 IOPS + 1 GB/s Amazon Elastic Block Store (EBS): $260/month
TOTAL: $275.18/month
This is now $255.18/mo more than ADTAQ's Storage 2 system. Unfortunately, AWS only supports up to 1 GB/s of throughput on their general purpose disk. So, this AWS instance is still $255.18 more despite having throughput capped at 1 GB/s and the ADTAQ system reaching 8.27 GB/s on one of the benchmarks. I guess that apples to apples comparison just flew out the window...
Clearly I'm biased, but in my opinion, this already shows enough value over AWS to make ADTAQ the obvious choice.
You may have noticed from the screen shot above that the ADTAQ VPS INCLUDES 6,000 GB of monthly data transfer for that same $20. AWS meanwhile has a cryptic "we charge for some traffic but not for others but for some others to certain geographic areas but not in certain directions just trust us and pay up" method of determining data transfer. Adding on 6,000 GB of outbound internet usage in a month to the AWS instance would look like this:
t3.small Amazon EC2 On-Demand instances: $15.18/month
2 TB + 16,000 IOPS + 1 GB/s Amazon Elastic Block Store (EBS): $260.00/month
Data Transfer: $552.96/month
TOTAL: $828.14/month
My Dad would be quick to tell you I am no math whiz, but my trusty calculator.exe tells me that this as-close-as-we-can-get-to-apples-to-apples comparison clearly shows that ADTAQ is a mind-numbing $808.14 less expensive PER MONTH than a system that is not even as fast disk-wise at AWS.
We just launched a new Custom Solutions offering for folks that need more than a single server and might want an internal network, firewalls, load balancers, etc. Yeah, it's not elastic or API driven, but if you know what you need ahead of time you are going to cut your costs dramatically; like 75% less, at least!
So, what do YOU think? Does this product have a future, or should I just give up because "omg no one can compete with Amazon"?
