Building A Virtualization Host

So what kind of hardware makes a good virtualization host?

Well, I would say the things you should consider are:

  • Processor, the more cores the better
  • Memory, the more the better
  • Storage, the more spindles the better

There are a couple more things you want to consider; for your CPU you really want a multi-core process that supports hardware virtualization.  I generally consider the Q6600, Q9300, and Q9400 to be good choices.  At $160, $170, $180 each you should probably consider the highest end processor your motherboard can support (some older Intel based motherboards may not accept the Q9300/Q9400 processors, so go with the Q6600, it’s about the same performance as the Q9300, just draws more power and produces more heat).  Of course, pricing varies — so figure out where the “sweet spot” is in the price curve for your favorite vendors.

With memory on a Intel processor you always want to populate all the banks; the interleaving will greatly increase the performance of your memory.  And with quality (Corsair DHX) memory costing only about $100 for 8GB (4x2GB) there’s really no excuse to scrimp.

Before I cover storage I’m going to throw in motherboards; yeah you need one of those two (as well as a case and power supply).  For motherboards I generally will pick something like the Intel DG33TL or DG45ID (the newer motherboard actually will cost less).  The built in video reduces the required component count, and you don’t really care about the video on your virtual host — it’s a server.  Also, that frees the x16 PCIe slot for a x8 RAID adapter if you choose to go that route (though there are now many motherboards that have multiple x16 PCIe slots that only cost a little more — but generally require a video card).  One other thing you might want in a motherboard is multiple ethernet controllers (and those should be Gig-E).  It’s not really required, for the most part your virtual infrastructure will be limited by your storage subsystem or your internet connection (depending on your application), but it is “nice to have”.

Now to storage.

The first thing is your system drive on a virtual host really isn’t that important.  Any reasonable SATA-II drive running in AHCI mode will be fine.  And you don’t really need to mirror it, since you can take a snapshot of it and restore it to another drive if it fails.  The decision of whether or not to mirror your OS drive will depend on other factors.  Also, since you definitely do not want your operating system swapping (no paging file) the performance of the drive isn’t a huge concern.  And if you want the ultimate, choose an SSD — that will let you boot very quickly, and there shouldn’t be any delay in writing a log file (32GB  is plenty, 16GB might be a little tight, but you could do it).

Your data drive, the one that will hold your virtual machine images, is very important.  For a small server you can start with just a mirrored (RAID1) or mirrored and stripped (RAID0+1) drive set of what ever size you need.  But understand that the number of spindles in your data set will greatly effect the performance of your virtual machines if they are disk bound (ie they read or write lots of data from the disk).  In fact, if any of the virtual machines are heavy disk users, they will impact all the virtual machines if you don’t have lots of spindles.

The rule of thumb I often use for virtual hosts that will have reasonable disk activity is the minimum number of spindles is the number of virtual machines plus one.  For economizing or for lightly loaded disk activity you can divid the number of virtual machines by two and add one… but you really always want at least a mirrored pair (single drives can get sluggish — they’re OK for a development workstation, but not a virtual server where you might depend on the machines running day in and day out).

To get more spindles you can use the pseudo RAID controllers build into the motherboard (the Intel Matrix controller isn’t bad, but it’s not a real hardware RAID controller); those controllers do fine for stripping and mirroring (and there’s no reason to buy anything more than that if that’s all you need).  If you get serious about virtualization and want to go with RAID5 (or RAID6) then you’ll want to invest in a real hardware RAID controller (and be careful when you buy, lots of entry level controllers actually aren’t any better than the Intel Matrix controller, except that they allow you to migrate to higher end RAID controllers seamlessly).

The absolute best name in RAID controllers is LSI, you can often save money buy purchasing an older series of controller, or a controller that uses the LSI chip set (but made by a systems vendor).  The most important thing is that the physical interface is SATA-II for the drives and PCIe for the system, and that the controller have enough channels for the number of spindles you’re likely to need (four, eight, twelve, sixteen, and twenty-four are the number you’ll see — LSI used to have a six channel, but they don’t offer that in the newer series).

You can buy a controller larger than you need, but you’re going to spend a great deal more money on the controller; and you’ll need a case and power supply that can handle that number of drives.

You will also want to consider hot-swap bays for more than four drives for sure; and those will add greatly to the cost of your machine (for 3.5″ drives you can get 1-in-1, 3-in-2, 4-in-3, and 5-in-3; for 2.5″ drives you can get 4-in-1… where this referes to the number of drives you can fit in a 5.25″ drive way).

What I do on my machines with sixteen channel RAID controllers is I have three 3.5″ 5-in-3 and one 2.5″ 4-in-1; the fifteen 3.5″ drives are attached to the first fifteen channels of the RAID controller (the sixteenth channel is unused), and four motherboard channels are attached to the 2.5″ drive bays.  My case actually has two power supplies, and all the 3.5″ drives are run off a 750W single rail power supply, and the 2.5″ and rest of the system are run off a 400W supply.  By using the combination of 2.5″ and 3.5″ everything fits nicely in the case, and the power supplies are more than adequate.  You’ll note that I use 2.5″ for system drives (I plan on going to SSDs eventually) — so obviously I’m not worried about the performance.  I have similar configuration for my eight channel RAID machines (except they don’t need dual power supplies). 

One final note, if you’re concerned about power consumption, you might want to consider building out a virtual host using only 2.5″ drives and stick with just using the motherboard controllers (real hardware RAID controllers consume a great deal of power).  With 500GB 2.5″ drives you could have 1TB mirrored and stripped, and that may be adequate for your needs.  Once you add a hardware RAID controller I’m not sure that you really need worry about the power consumption of the drives as much, but you’d need to do the math.

Originally posted 2009-02-11 01:00:16.