Hypervisor Shootout

Im a virtualization fanboy, sorry but its true. It solves so many of our problems (rollback of updates, power & aircon savings, operation resilience, zero downtime etc) and without any downside that I can think of. I use it at work and I use it at home – it rocks! Recently I ran into an issue with my home virtualization setup (I wrote about some of the issues here).

My home setup is pretty straight forward. The Host runs Centos minimal and upon that platform I use OpenVZ (great for thowing up and tearing down linux guests in a very short time) and VMware Server 2.0. I made the VMware choice a few years back, there wasnt much else out there and I wanted something that I could manage from a Linux machine – my home is entirely a Windows free zone 🙂 VMware Server 2.0 is managed from a web interface, so that works for me.

Anyway, with my recent issues I decided to re-evaluate my virtualization decision, thinking that I would probably just end up back with the same setup. Before I talk about which Hypervisors I have tried in the last week, lets look at my requirements

  • to be able to mange from Linux is high on the list but negotiable (I guess)
  • I need some kind of container technology for Linux guests (OpenVZ or VServer etc)
  • I need to be able to do full virtualization for guests that cant exist as containers
  • Id like to be able to quickly move a guest to another hardware platform easily, in case of maintenance or hardware failure.
  • Snapshots – to be able to take a snapshot and quickly roll back to it is pretty important to me
  • Ease of management
  • If its open source then great
  • Zero cost

So lets look at which solutions Ive tried

  • OpenVZ & VMware server 2 – its the current solution and works well for the most part
  • XenServer (running OpenVZ as one of the guests)
  • VirtualBox – I know, bet you didnt expect that
  • ProxMoxVE – an Open Source combination of OpenVZ and KVM

As you can see, Im a big fan of OpenVZ – its simple, fast, clean and offers quick guest migration to other hardware and very little overhead. The choices in that list, two maybe obvious and two maybe less obvious.

Lets start with the first option – VMware Server 2 and OpenVZ combination. Looking at my requirements you can see that it meets all except one (its not Open Source). Its a good combination and has served me well over the years. One of the requirements that I say it matches is being able to move a guess to another server easily. OK, this is a bit of a stretch as it would involve shutting down the guest, copying the files over to the other server and booting it up. A little bit of hassle but not bad. OK, lets move onto the next one.

XenServer 5.5.0 (using OpenVZ inside one of the guests). The install is pretty straightforward, once online you come to the first problem – management. XenServer is managed from a Windows thick client – there are 3rd party web front ends that you can install and that’s why I even entertained the idea. One of the upsides of this platform is that it offers live migration of all guests to another XenServer and that is included in the version that Citrix gives away for free. However, please not that live migration is only available if the CPUs of the Hosts are the same, which in my case they are not – so no live migration for me. I deduct a point for that, which you may think is a little harsh but my two home servers dont have identical CPUs. I ran this setup for a few days before deciding it wasnt the solution for me. Things that made me make that decision – no roll back to a snapshot. You have to take the snapshot, create a template from it and then create a VM from the template – all that in order to roll back to a snapshot! Thats madness!! Also, the fact that you cant over commit resources – so if you have 4 gigs of ram then your guests need to be configured so that they dont exceed this total. Weird, most hypervisors allow you to overcommit, anyway, so XenServer is nice but its no where near what I need. Live migrartion would be cool but it wont work in my setup. All that and I have to boot my laptop into Windows to manage it – no, XenServer you have been evicted.

Number three on the list is Virtual Box and I bet many of you are thinking that this is not a server solution, its a solution that runs on a Desktop. Well for those who dont realise there is a commanline tool called VBoxHeadless that lets you start guests from the command line and the thick client is available on Linux so its not as out there as you think. It does snapshots, its Open Source, it costs nothing etc but ultimately I decided against this option

So finally there is ProxMox. This is a combination of KVM and OpenVZ that all comes wrapped up on an install CD that uses Debian as its base. Again, the setup is pretty simple – you install onto the server and then all management is done via the Web Interface. (You also have ssh access to the host by the way). This solution is a great fit, it only misses out on one option, there are no snapshots available. As Proxmox uses OpenVZ I was able to import my existing OpenVZ containers into it easily and manage tem from the web interface from then on. For the sake of testing I installed an XP guest using the KVM option (fully virtualised) and it was all very simple and straight forward. This relies on you having a processor that it VT capable but I did, so that was no problem. In many ways Proxmox is better than my existing setup because all the guests are managed from one interface (before I used the VMware web interface for fully virtualised hosts and the commandline for OpenVZ containers). the other advantage this has is that its fully open sourced.

We have a winner

So who won, what did I decide to do? I have to say that even I was surprised by this as I had expected to go back to VMware but Ive actually stuck with the last option – Proxmox. OK it doesn’t let you create snapshots from the web interface but thats the ONLY thing it doesn’t do – and its all totally Open Source. Rock on!

Open Virtualization Format

A lot of this (moving from one hypervisor to another) wouldn’t be possible without the OVF, its great that so many people got behind it so quickly. The last thing we need in the virtualization space is vendor lock in. In short you can export your Virtual Machines into OVF format and from there into your new hypervisors format. Most modern hypervisors will support this. Interestingly, while VMware ESX(i) supports it, VMware Server 2 didnt – but VMware provide OVFtool for this very purpose. Citrix provide a link about how to convert VMs here. VirtualBox not only supports importing and exporting from OVF directly but, as I recently found out, can directly read VMware disk formats and let you boot a VM without the need to convert.

I’d love to hear your experiences in this area, drop me a line via a comment or Contact form

OSG

UPDATE: Just a quick note to add that Virtual Box have just announced that their new version will introduce “teleportation” of a live guest from one platform to another – ala VMware Vmotion but VirtualBox can do it to disparate hardware – even different CPUs (AMD to Intel). Read more here

8 Replies to “Hypervisor Shootout”

  1. Great write-up and thanks for following up your previous posting on virt technology. We’re currently evaluating several virt technologies so this is valuable info for us. I’d heard of KVM of course, and OpenVZ kinda vaguely but ProxMox seems to have brought the two together. I did not hear about ProxMox until reading your post. I’m gonna add ProxMox added to the bunch we’re currently evaluating. One thing, the screenshot here appears to show that snapshots via web interface is possible: http://pve.proxmox.com/wiki/Backup_-_Restore_-_Live_Migration – is this true?

    BTW a Debian developer who has recently joined our company has been waxing lyrical about KVM and I’m thinking it is now _the_ way to go for future Linux-based virt.

  2. Excelente review! I use KVM as my main virtualization platform for non-production systems. At work we use Xen but we’re evaluating new platforms with better tools (as it’s just paravirtualized Xen, not XenServer).

    I’ve never heard of ProxMox before but it sound really cool. I will give it a shoot and let you know what I think of it.

    Congratulations again!

  3. One correction – Proxmox VE absolutely has snapshots. It uses lvm for the backend. The snapshots have to be done via the qemu command line. Its under the “Monitor” tab on the web interface. To take a snapshot, type “savevm “. To load a snapshot, type “loadvm “. In my experience, the snapshot save/restore mechanism is faster than other Hypervisors.

  4. Hello,

    I really appreciated this article. I was evaluating VMWare ESXi for home use. It works quite well but I have a preference for someting that is completely open source.
    I will definitely be looking at Proxmox VE.

Leave a Reply

Your email address will not be published. Required fields are marked *