kernel panic a developer's point of view

I have experience with amazon’s AWS cloudservices for years now, but still have problems with EBS (=amazon network storage) performance.  It is not very fast and if you just want to run a simple LAMP server it requires a lot of work and experience to set it up the right way. But how fast is it? How does it compare to the offerings from Linode and the new kid on the block: digitalocean?  Both have their advantages: Linode has it’s standard 8 core architecture and RAID. digitalocean has fast SSD disks. But how do they stack up against amazon?

Compare same-priced offerings.

To compare apples with apples instead of oranges, I decided to test the following offerings:

– Amazon Small instance –  (with high-utilisation reserved instance) $24 per month.

– Linode 1Gb plan – Costs $20 per month

– Digitalocean $20 plan.

What do they offer for this price?

Amazon: 1.7 GiB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of local instance storage. But the local storage can not be used for permanent storage, as it is ephemeral (destroyed upon server termination). So you need additional EBS storage attached to your instance, which adds to the cost. Also this is with only 1Gb outgoing data transfer, additional data costs $0.12 per Gb. EBS storage is $0.10 per Gb per month, so add $2 per month for 20Gb of EBS.

Linode: 1Gb of memory, 24 GB RAID Storage, 8 cores with lowest priority, 2Tb data transfer included. additional data at $0.10 per Gb

Digitalocean: 2G memory, 2 cores, 40Gb SSD storage, 3Tb transfer included, additional data transfer $0.02 per Gb

Note that amazon can cost a lot more if your data transfer is higher. If you really need 2T outgoing traffic, amazon bills you a hefty $240 additional cost.

Benchmarking

For this test I used Amazon Linux (fedora style distro) for EC2 and Centos 6.4 for Linode and DigitalOcean. Apache 2.2.x, php 5.3.x, mysql and ApacheBench (ab) on a separate client machine to saturate the server.

I requested a php page that shows a calender of events, which it fetches from the database, does some layouting and shows the result. It is build upon Laravel 3.

I did multiple AB tests.

1. 100 requests and a concurrency of 10. (ab -n 100 -c 10)
2. 500 requests with a concurrency of 10 (ab -n 500 -c 10)
3. 1000 requests with a concurrency of 20 (ab -n 1000 -c 20)
4. 2000 requests from 2 different machines each with a concurrency of 40. (ab -n 2000 -c 40 on both machines at the same time)

The tests were run from an independent server hosted by another company.

These are the results:

During the tests I run top on each server and wrote down the highest server loads for the bigger tests.

Conclusion

Well, amazon just sucks. At only 4.5 request per second it is about 15 times SLOWER than the linode box. Is this due to the slow EBS volumes or is this due to their 1 ECU compute unit?  Looking at the high load figures, the problem seems CPU-bound in amazons case.

Digital ocean is about half the speed of Linode, in spite of their fast SSD disks. But they are stilll 6 times faster than amazon.

But the winner without any doubt is Linode. Their 8 core server really rocks in this case.

Of course each of these providers offer larger instances at a higher cost. But this test was designed to target the $20/month price point.

I have to rethink my amazon strategy. Is amazon overpriced? do I need expensive high-CPU instances to compete with the linode performance? It seems linode has the best cards in this space currently, and the differences are too big to ignore.

Update may 2014

Since a few weeks Linode changed their offering to SSD, faster CPUs, double Ram but you get strict core isolation now. You now get 2 cores for $20 instead of 8. So I upgraded (?) my linode servers and did the test again:

The previous linode machine had far better result, just because they did not have strict isolation. I just took over the whole machine and became a “noisy neighbour” for my fellow tenants as Evan pointed out (see comments). The test is now more realistic, and the linode results are now comparable with digitalocean. But still amazon is the slow kid on the block. I agree that amazon has a lot to offer, but their offerings are overpriced. And while linode upgrades their hardware at a pace of about once a year (the $20 offering went from 512Mb to 1G to 2G, got SSD and new CPUs the last 2 years), Amazon is still using the same hardware that they offered years ago. Did they lose their edge to innovate?