Build, buy, or upgrade for ESXi 5 lab?

MacGuffin

I finished my VMware vSphere 5 class last week and I've been doing some research in creating a lab to get some real world practice before taking my certification test. I took a look at what computers I have on hand and found myself coming up short. I have numerous 32-bit machines that, while perfectly functional, cannot run the 64-bit ESXi hypervisor. These machines can be used as the VM file servers, host for the management client, and other tasks but they cannot host the hypervisor itself.

I do have the following two 64 bit computers at my disposal:

- MacBook Pro, 2.2GHz Core2Duo, 4GB RAM, 500GB SATA
- Dell PowerEdge SC440, 3.0GHz dual core Pentium D, 2x 160GB SATA

Neither are particularly suited for ESXi as they are since in their current state they are unsupported but I suspect I can make one of them work with the proper combination of software and hardware.

With that background I'll ask you, should I...

- Upgrade my current hardware?
- Build a new PC?
- Buy a new PC

Since feel quite ignorant of the current state of price/performance ratios, what current computer offerings are out there, etc. I won't list what I see as "pros" and "cons" but rather "pros" and "fears". I don't know what the cons are but I can suspect what they might be. My fears may be unfounded and end up not being an issue. I have not bought a new computer in over five years and I have not built a computer from parts in over 10 years. I'm not much of a gamer so I have not felt a need for high performance hardware. Also, being the "computer guy" in my extended family I have been handed down considerable amounts of non-working, half working, or mildly outdated hardware with which I've been able to piece together a fairly large network/lab in my basement. I also have many spare parts lying about, much of it is dated.

This is a bunch of me thinking out loud. If this is too much information then skip to the summary.

The options as I see them follows with what I see as the pros and what I fear as cons.

Upgrade MacBook Pro:
Pros:
- Appears to be low cost. VMware Fusion sells for about $50. Bringing the RAM up to 6GB would be between $50 and $100. Perhaps an OS upgrade, about $30.
- Should be simple. I should just be able to install the above listed upgrades and get to work.
Fears:
- Slow. This laptop is my workhorse, I use it everyday for web browsing, e-mail, Skype, etc. This means I'll have many programs running at the same time. It already runs hot and gets sluggish at times. Even after a memory upgrade I fear the computer will become nearly unusable with another OS running in a VM, having been bogged down with so much stuff running.
- Inconvenient. Having used other emulation software I've found that using a wireless connection for a virtual machine just does not work (abstraction issues or something), or is spotty (connection drops intermittently). This would mean being tethered to my network for the hypervisor to work reliably. There is also the general inconvenience of using a laptop as a workstation, limited screen size, cramped keyboard, limited expansion and/or tethered expansion (USB, Firewire, network, or other wires hanging off of it).
- It won't work. There is the possibility that even after I've gone through this trouble and expense that I missed something very important and I will not be able to get ESXi running as I'll need it.

Upgrade Dell server:
Pros:
- Low cost. New processor would be between $50 and $100. (Current processor does not have virtual machine support extensions.) Memory upgrade (from current 2GB to 4GB max supported) would be between $50 and $100.
- Some room for expansion. I've got plenty of spare hardware lying around to plug in, PCI cards, hard drives, USB adapters, etc. If I need to plug in something I don't already have I should be able to purchase it cheaply.
- Convenient. It's not really doing anything right now and it should be able to do everything it did before with the proper application of the right virtual machine and/or hardware. It's also current sitting right where it would need to be, connected to a KVM switch, ethernet switch, etc.
- Should be simple. What could possibly go wrong?
Fears:
- I'll kill it. I've done processor upgrades before and I don't have a very good batting average of getting a working result. Sometimes it's just a matter of the computer not working to expectations. Sometimes I end up with a dead motherboard. This is a very nice computer, it's relatively powerful and has shown itself to be reliable. I'd hate to trash it.
- It won't work. This is really just a restatement of the above fear. Even if the computer remains functional after my upgrade there is a possibility (like with the MacBook Pro upgrade possibility above) where I'll go through the time and money and end up missing a vital detail and the computer will not run ESXi, will run it very slowly, or only with more time and expense.

Build new:
Pros:
- Should be only moderately more expensive than the other upgrades. I have spare parts lying around, power supplies, cases, keyboards, hard drives, PCI cards, etc. I should only need a CPU, motherboard, memory.
- Should be fast, functional, and relatively simple. If I do my research right everything should just plug together and work. With new parts at the heart of the computer it should run circles around anything I currently own.
Fears:
- Cost. Will I be saving any money? There's a lot of unknowns here. Last time I built a computer from all new parts I found that I saved no money, ended up with no real support (everyone I called pointed the finger at the vendor of some other part I bought), and (because I didn't do *enough* research) wound up with a computer that was mildly crippled (some software did not run).
- I'll kill it. I've killed computers before by plugging in parts. None of computers I killed before had much real value in dollars but they had some value to me in time and emotion invested. New parts are less likely to fail but they will have a more tangible monetary value. This is really just another way to express the cost (tangible and intangible) I could expend in this path.

Buy new:
Pros:
- Simple. Find a computer supported by VMware vSphere, buy it, move on.
- Fast. Any new computer will be much faster and more capable than anything I have now or could piece together.
- Leverage current assets. My laptop, server, and other computers will remain unchanged and can only add to the network I'm building.
- Low risk. If the computer is DOA, fails within warranty period, or does not work like I want it to then I should be able to return it, get someone on the phone to help me out, etc.
Fears:
- Cost. This is the most expensive option. Will I spend too much? Was there a cheaper computer that I could have bought?

Summarizing:

Help me figure out the cost/benefit here. Cost is a factor and I don't have enough experience to really figure out what I need and how much I must spend to get it. I'm leaning towards two paths here. The first path I'm considering is an upgrade path. Start with upgrading my laptop. If things don't work out like I want I will (or should) still have a working laptop only with more memory and software. If the laptop upgrade does not work like I want then I can upgrade the Dell. Lather, rinse, repeat until I get something working or I've gathered up enough parts to build a new computer.

The second option I'm considering is to just buy a new computer right off. I've seen how-tos on getting Mac Intels to run ESXi. I can buy a new Mac and either get the new Mac to run ESXi or turn my current laptop into the hypervisor host. I'd buy a Mac only because that is what I am most familiar with, I know they support VTx, and I know they will run Mac OSX Server, Windows Server, Linux, or just about any other OS I can foresee wanting to experiment with in the future. Maybe I'm the victim of selection bias but I've rarely seen Apples go bad. People will give me their broken computers to fix or dispose of. The only dead Apple I got was an aged iMac with a bad video capacitor (it was a CRT model, the capacitor was almost the size of a beer can) at a time when there was a rash of bad capacitors going around.

I've seen new, used, and refurbished Macs in the $500 to $1000 range that should work for me. If any other option gets in that range then I'll just get another Mac. I'd very much like to keep this below $300 if I can.

ptilsen

I only skimmed your post, but my answer is buy. Get used PowerEdge or Proliants on Craigslist. You can build or upgrade cheaper, but ultimately it's going to be quicker, easier, and less stressful to just buy them. Use that time and energy on your studies instead of your hardware. Unless you really need the hardware experience for some reason, just buy it all.

MacGuffin

ptilsen wrote: »

I only skimmed your post, but my answer is buy. Get used PowerEdge or Proliants on Craigslist. You can build or upgrade cheaper, but ultimately it's going to be quicker, easier, and less stressful to just buy them. Use that time and energy on your studies instead of your hardware. Unless you really need the hardware experience for some reason, just buy it all.

Yes, I did go on for a bit there. I'll see if I can be more to the point this time.

I agree that while money is valuable so is my time and sanity. I'm leaning toward buying a pre-built computer but I have very little experience in what to look for. You mention the PowerEdge, I have one and I like it but it does not support VT. Dell seems to make solid gear and I would not be opposed to buying another. One concern I have is the wide variety in processors, chipsets, and general features that Dell puts under one part number. Take the SC440 I have for example, this computer can come with one of (IIRC) four different processors. I happened to get the one that does not support VT. How can I know the particular Dell I'm looking at supports VT? Is there a particular model I should look for? Is there a model I should avoid?

In general what should I expect to have to spend for a Dell, HP, or whatever that will run ESXi? I'm not terribly concerned about the hardware getting the official VMware seal of approval, but I do want to make sure that ESXi 5 will run on it before I buy. Is there some kind of shortcut or rule of thumb I can follow to quickly sort out the computers that will work for me from the ones that will not?

A bit of quick math tells me that upgrading one of my existing computers to run ESXi 5 would likely cost $200 to $300. If I can find a brand new computer that runs ESXi 5 for less than $600 then it's probably worth it.

One more thing that just came to mind. If I am forced to run ESXi inside an emulator for some reason is there an emulator besides VMware Workstation that will allow this?

MentholMoose

My vote is to upgrade your MBP and run ESXi as a VM in VMware Fusion. 6GB is probably cutting it too close for comfort (especially if it's a multi-purpose machine) so upgrade to 8 GB. RAM is dirt cheap, so replacing your existing RAM with 2 x 4 GB SO-DIMMs should be around $50 (for DDR3, might be a bit more for DDR2) from a reputable 3rd party manufacturer (e.g. Kingston, Crucial, etc.). With 8 GB RAM you should be able to run two ESXi VMs, a vCenter VM, and a storage VM with no problem. The 500 GB hard disk might slow things down, though, in which case consider getting an SSD (a 240 GB SSD is $250-300 and generally enough for a multi-purpose machine... or if you can live with less space you can save some money). In my experience, an SSD helps tremendously when doing labs on a multi-purpose laptop/desktop.

For the VCP 4 and 5 I used several older Supermicro servers with Core 2-based Xeons. I liked having physical servers but there are downsides. Specifically, they are noisy, take up a lot space, use a lot of electricity, and generate a lot of heat. When I was studying for the VCP 4, my power bill went up $50 a month. For those reasons I wouldn't recommend buying or building older servers unless you are going to get rid of them when you're done. Overall it is not all that practical, and having a beefy laptop is better for most purposes... my primary machine is a ThinkPad with a Core i5 CPU, 8 GB RAM and a 160 GB SSD, and I can do most labs on it with no impact to anything else I'm doing.

Good luck on the exam.

MacGuffin

MentholMoose, my MBP does not support more RAM than 6GB. Upgrading to 8GB is not an option.

Is this about right?
$50 RAM
$300 SSD
$50 VMware Fusion
===
$400

I took a look at the Apple website and I can get a refurbished Mac Mini for $519. This would have 2GB RAM and a 500MB drive. Not fancy but I'd be able to run on the metal, reducing RAM needs. The processor is a 2.3GHz i5, a wee bit faster than my MBP.

For $699 I can get the refurbished Mac Mini with 4GB RAM, 2.5 GHz i5, and 500 GB drive. A bit more memory, a bit more processor, a bit more $$$. Perhaps a bit too much $$$.

I don't know, a Mac Mini looks like a pretty good deal. I'm not even sure I can find the memory as cheap as you claim for my MBP. I went to a web site I trust and they sell the 4GB SO-DIMM for $80 - $90. You say that DDR2 would be a bit more? Guess what? My MBP uses DDR2 RAM. That puts your MBP upgrade estimate closer to $450.

Same website sells the Mac Mini RAM (DDR3) at $30/4GB (one DIMM), $50/8GB (two DIMM kit). That refurbished Mac Mini with the 8GB RAM upgrade comes in at $570 or so.

Leave out the SSD upgrade for the MBP and I'm back to $90 for RAM (single 4GB DIMM, upgrade to 6GB), $50 for VMware. $140 total.

I tried installing ESXi 5 inside VirtualBox 4.1 tonight. It hung on the install. In the morning I'll play with some settings and try again. I'll probably end up buying VMware Fusion anyway just to get started. It could serve as a fall back if I get a Mac Mini and cannot get ESXi to install on the metal for some reason.

Anyway, I got some numbers to ponder. Not set on the Mac Mini, it's just something I am familiar with to use as a reference point. I'm still looking for viable alternatives.

Akaricloud

I was in pretty much the same situation and decided to buy new.

From dell business and education outlet I bought a Optiplex 790 for $480 after tax then added a SSD and upgraded the ram.

Total was around $600 for a i7 2600, 16gb ddr3 and 120gb agility 3 (plus I kept the 500gb it shipped with). It runs ESXi 5 flawlessly and I've yet to see any slowdowns while running 4 VMs of server 2k8 and one Windows 7 VM.

MentholMoose

MacGuffin wrote: »

MentholMoose, my MBP does not support more RAM than 6GB. Upgrading to 8GB is not an option.

If that is the case, I'd recommend against upgrading since 6 GB will be limiting and the Core 2 Duo is getting pretty old.

MacGuffin wrote: »

I took a look at the Apple website and I can get a refurbished Mac Mini for $519. This would have 2GB RAM and a 500MB drive. Not fancy but I'd be able to run on the metal, reducing RAM needs.

To pass the VCP you need extensive experience with a vSphere environment with multiple ESXi hosts. If you think you can get away with a single ESXi host for studying, which may be fine if you have already enough vSphere experience, save your money and run ESXi as a VM in VMware Fusion since running it on physical hardware won't be much better. I don't think a Mac Mini is appropriate for a lab machine since it is not cost effective and it lacks expandability. You can get a lot more for your money with an Optiplex as Akaricloud suggested. On that Optiplex (or even a lesser one) you could install VMware Workstation and run a full vSphere lab.

MacGuffin wrote: »

I tried installing ESXi 5 inside VirtualBox 4.1 tonight. It hung on the install. In the morning I'll play with some settings and try again. I'll probably end up buying VMware Fusion anyway just to get started. It could serve as a fall back if I get a Mac Mini and cannot get ESXi to install on the metal for some reason.

Here's an article explaining how to install ESXi 4.1 in VirtualBox. You could try following it to see if it applies to 5.0.
Setup ESXi in VirtualBox | Steven-Barrett.co.uk

VMware Fusion is known to work.
VMware KB: Installing ESXi 5.0 in VMware Fusion

MacGuffin

Akaricloud wrote: »

I was in pretty much the same situation and decided to buy new.

From dell business and education outlet I bought a Optiplex 790 for $480 after tax then added a SSD and upgraded the ram.

Total was around $600 for a i7 2600, 16gb ddr3 and 120gb agility 3 (plus I kept the 500gb it shipped with). It runs ESXi 5 flawlessly and I've yet to see any slowdowns while running 4 VMs of server 2k8 and one Windows 7 VM.

That sounds like a good price, much cheaper than what I saw on Dell.com. Problem is that I live in eastern Iowa, we don't have Dell outlet stores here. We got Best Buy, Sam's Club, a few small mom and pop computer shops, and whatever we can find online. Your price sounds like the exception rather than the rule. I was unable to find a comparable computer online for a similar price while still having some manufacturer support. Feel free to prove me wrong.

Doing some research I found that there are some Intel i3 and i5 processors that do not have VT but all i7 processors have VT. Does that sound right? With that in mind I may want to confine my search to computers with an i7 processor.

Apple is selling a refurbished Mac Mini Server for $849. This has a quad core i7 @ 2.0GHz, 4GB RAM, dual 500 GB drives, MacOSX Server. New it's $999.

Dell sells an Optiplex 990 for $939. It has an i7 processor @3.4GHz (does not say how many cores, dual?), 4GB RAM, single 500 GB drive, DVD-RW, Windows 7 Pro.

Seems like a toss up to me. Should I flip a coin?

It's more than I planned to spend but it looks like I might have to spend that much to get what I want.

How helpful would that Windows 7 license be to me? Does the VMware vClient (or whatever the ESXI management program is called) run on 32 bit Windows XP? I got plenty of XP computers lying around, I'd prefer to run the software off of the hypervisor host for both realism and to reduce load on the host.

MacGuffin

MentholMoose wrote: »

If that is the case, I'd recommend against upgrading since 6 GB will be limiting and the Core 2 Duo is getting pretty old.

I tried again with VirtualBox host and ESXi guest. Brought my laptop to a crawl. I went to get lunch while the installer loaded, still didn't finish by the time I got back. Also, Core 2 processors do not support nested virtualization. Even if I got ESXi running as a guest I doubt I'd get the ESXi guest to load a guest of it's own.

If I'm going to use my existing laptop to run ESXi it appears I'd have to install it on the metal. That doesn't work for me.

MentholMoose wrote: »

To pass the VCP you need extensive experience with a vSphere environment with multiple ESXi hosts. If you think you can get away with a single ESXi host for studying, which may be fine if you have already enough vSphere experience, save your money and run ESXi as a VM in VMware Fusion since running it on physical hardware won't be much better. I don't think a Mac Mini is appropriate for a lab machine since it is not cost effective and it lacks expandability. You can get a lot more for your money with an Optiplex as Akaricloud suggested. On that Optiplex (or even a lesser one) you could install VMware Workstation and run a full vSphere lab.

Here's an article explaining how to install ESXi 4.1 in VirtualBox. You could try following it to see if it applies to 5.0.
Setup ESXi in VirtualBox | Steven-Barrett.co.uk

VMware Fusion is known to work.
VMware KB: Installing ESXi 5.0 in VMware Fusion

Thanks for the links. Looks like either VirtualBox or VMware Fusion would work for me if for some reason I'd have to run MacOSX as a host. This would require a Core i (or is it "iCore"?) processor for the ESXi guest to host VMs of its own.

I guess I didn't give much thought to the need for multiple ESXi instances. I knew I'd have to do it, just didn't think much about how I'd do it. Your right that I'd need to run multiple ESXi instances. To do this I'd need nested virtualization or multiple computers. If I must I might still be able to run ESXi on my laptop to fulfill the multiple ESXi instance requirement.

Your comment that a Mac Mini would not be appropriate as a lab computer confuses me. What is it about the Mac Mini that's lacking? Sure the Dell Optiplex has two PCIe slots but what would I need them for? I'm not going to put any video cards in it. Both computers have two hard drive spots. From where I sit I can see at least three Firewire/USB external DVD drives. Any storage beyond that is going to be on the network.

I suppose more network connections might be nice but this is a lab, not a production environment. I can emulate multiple network connections in software. If I need more real ethernet ports for some reason I'd likely end up getting some cheap USB to ethernet adapters rather than PCI or PCIe cards anyway. As mentioned before I have a box full of PCI network cards but these cheaper Dells do not have PCI, they have PCIe. I also have a couple USB to ethernet adapters already.

Your comment on saving money by running ESXi in VMware Fusion also confuses me. Perhaps you were assuming I'd run VMware Fusion on my laptop. From my research it appears that I will not be able to run ESXi on my laptop because of its hardware limitations, it cannot run nested VMs and it maxes out at 6GB RAM.

It was my intention to avoid running ESXi on top of a host OS to reduce RAM requirements. Without a host OS I could probably save one GB in RAM. It would also save me $50 from not buying VMware Fusion.

I'm leaning toward the Mac Mini Server at the moment. I know it has VT and supports nest VMs. The Dell website is a bit fuzzy on what model contains what processor, it does not even list how many cores are on the die. The extra drive space and MacOSX server are a nice little bonus. I'd like to have a Windows 7 license but its not a killer feature, I'm quite likely going to wipe the drive clean anyway to install ESXi. Will the Dell recovery DVD allow a recovery of Windows 7 inside a VM?

Akaricloud

MacGuffin wrote: »

That sounds like a good price, much cheaper than what I saw on Dell.com. Problem is that I live in eastern Iowa, we don't have Dell outlet stores here. We got Best Buy, Sam's Club, a few small mom and pop computer shops, and whatever we can find online. Your price sounds like the exception rather than the rule. I was unable to find a comparable computer online for a similar price while still having some manufacturer support. Feel free to prove me wrong.

Doing some research I found that there are some Intel i3 and i5 processors that do not have VT but all i7 processors have VT. Does that sound right? With that in mind I may want to confine my search to computers with an i7 processor.

Apple is selling a refurbished Mac Mini Server for $849. This has a quad core i7 @ 2.0GHz, 4GB RAM, dual 500 GB drives, MacOSX Server. New it's $999.

Dell sells an Optiplex 990 for $939. It has an i7 processor @3.4GHz (does not say how many cores, dual?), 4GB RAM, single 500 GB drive, DVD-RW, Windows 7 Pro.

Seems like a toss up to me. Should I flip a coin?

It's more than I planned to spend but it looks like I might have to spend that much to get what I want.

How helpful would that Windows 7 license be to me? Does the VMware vClient (or whatever the ESXI management program is called) run on 32 bit Windows XP? I got plenty of XP computers lying around, I'd prefer to run the software off of the hypervisor host for both realism and to reduce load on the host.

Check out Dell's business and education outlet site. They are about 30% off to begin with and have additional 20-30% off deals almost every week. Like I said, I got mine, with higher specs than what you posted, for under $500. Right now without any of their deals I see quad core 3.4ghz i7 optiplex 790s for $600.

Google the processor, I'm sure you can determine how many cores it has but it's most likely 4.

MacGuffin

A lot has happened in the few hours since I last posted. The short of it is that I'll be putting this project on hold for a while.

A bit longer story is that I got a hold of some more information. Most importantly is that I got some leads on a new contract. If this works out I can save some more money and buy a computer I want rather than just something to get me by. Also, I've found out that the VCP5 practice tests that were promised to me by my learning center will not be available for more than a month. I also did some checking on when the next test will be offered and none look to be available in the area soon.

With no lab computer, no practice test, and (more importantly) no certification test I believe I should divert my efforts in a more productive direction. (Before anyone even asks, no, I'm not driving to Chicago to take the certification test. Have you seen the price of gas today?)

I did do some more research before coming to the conclusion to put this project on hold. My brother got me in touch with a friend of his that builds computers as a side job. After a short conversation on my needs he gave a few suggestions on what I should look for and agreed to build a computer for me at cost as a favor. He e-mailed me a part list for a pretty sweet system for about $600. The price did not include some parts that I told him I could provide myself. This experience only reinforced my estimation that building my own PC was not going to save me enough money to be worth the hassle. I've found some information that tells me the Optiplex 990 or the Mac Mini Server will provide enough extra features (like Wi-Fi and smaller footprint) and a high probability that ESXi 5 will run with the only possible glitch being that of a missing NIC driver (with plenty of remedies for the glitch) to make either one a more preferable choice. Oh, and no assembly required!

The original plan from a week or so ago was to put my Cisco certifications on hold and get the VCP 5 certification done while the training was fresh in my mind. Now the plan, with what I learned today, is to return to my Cisco studies for the next few weeks. I'll plan on taking the ICND2 test next week and the CCNP Route test next month. By then I should have more to work with for my VCP5 studies.

Thanks all for the replies to my questions, you have all been very helpful. I will use the information in the future. I'm not going anywhere so feel free to continue the discussion if you like. I've subscribed to the thread so I'll see any updates in my e-mail. I just don't plan on acting on this project for at least a month.

MentholMoose

MacGuffin wrote: »

I tried again with VirtualBox host and ESXi guest. Brought my laptop to a crawl. I went to get lunch while the installer loaded, still didn't finish by the time I got back. Also, Core 2 processors do not support nested virtualization. Even if I got ESXi running as a guest I doubt I'd get the ESXi guest to load a guest of it's own.

Using a Core 2 Duo only prevents running nested 64-bit VMs (32-bit is fine). If it is not working well on your laptop, I don't think it is necessarily the CPU. I have a desktop with an E8400 Core 2 Duo, 8 GB DDR2, and an 80 GB Intel SSD, running Windows 7 64-bit, so I did some testing with Workstation 8. I built two ESXi 5.0 VMs (2 GB RAM each), a Fedora 16 64-bit VM (768 MB RAM) for an NFS datastore, and a Windows 2003 64-bit VM (2 GB RAM) running vCenter. It works fine. Nested 32-bit VMs run and can be live migrated. As far as the VCP goes, the main limitation of not having nested 64-bit VMs is the inability to do VDR labs, and having only 8 GB RAM might be somewhat of a constraint for a vSphere in a box solution using VMware Workstation.

MacGuffin wrote: »

Your comment that a Mac Mini would not be appropriate as a lab computer confuses me. What is it about the Mac Mini that's lacking?

When it comes to labbing, the Mac Mini doesn't seem very cost effective (based on the prices you mentioned), and it lacks expandability relative to other systems in the same price range. I didn't know it could take two hard disks, which should suffice for a vSphere in a box solution, but the RAM seems to be limited to 8 GB (I checked the Apple Store). Ideally I'd want 16 GB RAM, especially at this price point. Labbing primarily requires disks and RAM so those are what I'm concerned with. The CPU performance hardly matters at all when labbing and the CPU just needs to support certain features, so spending extra for a top of the line CPU won't help much, if at all.

I have a desktop with a AMD E-350 CPU/mobo combo and an 60 GB OCZ Agility SSD ($75), so I did some testing. The E-350 is a low-end, low-power CPU, but I had no problem installing ESXi 5.0 as a VM in VMware Workstation and running nested 64-bit VMs. I can't attempt a full vSphere environment (it's running Windows XP and only has 4 GB RAM), but it can take 8 GB, so I think this machine with 8 GB RAM and a 64-bit host OS would be a decent vSphere lab. A barebones E-350 machine is cheap (Newegg has one from Foxconn for $135), so you could just add an 8 GB DDR3 kit ($35) and an SSD and probably be set. Unfortunately it doesn't seem to support 16 GB RAM (2 x 8 GB DDR3 DIMMs are only $100).

MacGuffin wrote: »

Your comment on saving money by running ESXi in VMware Fusion also confuses me. Perhaps you were assuming I'd run VMware Fusion on my laptop. From my research it appears that I will not be able to run ESXi on my laptop because of its hardware limitations, it cannot run nested VMs and it maxes out at 6GB RAM.

To clarify, my suggestion was if you only need one ESXi host, running it as a VM on your MBP may be an option. If your MBP can't run one ESXi VM well, or you want to run multiple ESXi hosts, this option can be ruled out.

MacGuffin

MentholMoose, it appears to me that we have come to the same conclusion on my MacBook Pro just by different paths. I won't bore you with my entire thought process but I basically boils down to that my MBP can be either my daily workhorse or my lab in a box, just not both. It just does not have enough RAM and processing power and cannot be upgraded to have enough of either.

Since I still need to retain the capability of my MBP and I'd need something that can run ESXi 5, not necessarily as a lab in a box but enough VMs that I can make up the rest of the lab with other hardware I own. My research lead me to believe that it made much more sense to invest in a computer capable of being a lab in a box than try to host ESXi on my MBP and try to find a replacement for it as my daily workhorse.

When it comes to the Mac Mini I've found that it will support up to 16GB RAM. Apple does not sell that configuration as a build to order option, allowing only 8GB as a max configuration from their store. I'll leave all the theories as to why Apple does this as an exercise for the reader.

Perhaps the difference in the perception in value of the Mac Mini comes from a difference in the perception of the value of our own time, the different perception of the value of a known quantity, and a difference in local supply and demand dynamics. I am not aware of places around here that will sell used and discontinued hardware cheap. You may know of places online that will sell this stuff cheaper than what I've found, if so then please share.

I've found numerous websites that explain the process of installing ESXi on MacBooks and Mac Minis. This gives me great confidence in success. I also know that I still have the possibility of running ESXi inside VMware Fusion or VirtualBox. By not running on the metal I have a cost in speed and memory but I at least *know* it will work. I already got bit with the Dell server I have, it won't run ESXi but another variation on the same model will. My confidence in successfully getting ESXi to run on the next Dell, HP, or whatever has been bruised. Perhaps it's misplaced but I don't have the funds to experiment.

I've seen the listings online for different models of new and used equipment, some will be advertised as "with VT" which is what I need. Some will not advertise that, they might have it but I don't know that. Without knowing what to look for I cannot determine if these cheaper computers will do what I need so I pass them by, not willing to gamble with my limited funds.

You mention the value in having RAM. I have not experimented with ESXi 5 outside of a classroom so I'm still fuzzy on what I really need. Perhaps I've been placing too much focus on the processor and not enough on the RAM. How much RAM do I need? Or rather, with a given amount of RAM what can I expect from that computer? If I had to choose between gigahertz and gigabytes then what kind of calculus should I perform to determine the relative value?

Other than my own "toys" in the basement I have not built a computer for myself in something like 15 years. In the mean time I was not concerned about cost since someone else paid for it. Either the hardware was from work (I didn't see the receipts, I just put it together) or they were given to me from friends and family as "junk" or as payment for services rendered. It seems I have much to learn before I invest much more time and money in this.

reppgoa

I JUST built a new ESXI lab. I went with an I7 2600 (not2600k) for the VT-D feature. I have 32gb ram, and 2 120gb SSD's with background garbage collection (no trim needed) in raid0. I found Tinkertry to be a very useful resource when building my lab. Take a look at his "vZilla" build.

reppgoa

This blog was also super useful:

Building the Ultimate vSphere Lab – Part 1: The Story « Boerlowie's Blog

dales

Sorry bit late to the party on this one but I had 2 dell sc440's I've used very easily for my vcp 3 and 5, abd by the way dell "supports" 4GB but the motherboard takes 8GB. Both of mine had 8GB of crucial memory but with the price of DDR2 these days I build a whitebox pc with 16GB in the end and use vmware workstation now instead. Much easier and less cables which pleases my wife grately!

MacGuffin

reppgoa wrote: »

This blog was also super useful:

Building the Ultimate vSphere Lab – Part 1: The Story « Boerlowie's Blog

I agree, that's some good info. I've read it and picked up a few things. One thing of note is it verified what I heard elsewhere, the Intel i7 is a good processor to have. It will have multi-threading and the VT extensions making it a no brainer for virtualization labs.

One point I may argue with him on is the "need" for 16GB RAM. He's running ESXi on top of VMware Workstation on top of Windows 7, that adds considerably, IMHO, to the RAM used. I believe I could get away with half that if I ran Windows Hyper-V Server or ESXi 5 on the metal instead. I'd also consider running the iSCSI, AD, SQL, and/or other servers on separate physical servers to further save on RAM. My home network is not wired for gigabit (yet) so the speed gain may be debatable.

This is the Nth time I've seen someone call for the use of SSD drives for the boot drive. I'm still not convinced this is as critical for performance as people claim. I'll do some reading on this later.

Thanks for the link.

MacGuffin

dales wrote: »

Sorry bit late to the party on this one but I had 2 dell sc440's I've used very easily for my vcp 3 and 5, abd by the way dell "supports" 4GB but the motherboard takes 8GB. Both of mine had 8GB of crucial memory but with the price of DDR2 these days I build a whitebox pc with 16GB in the end and use vmware workstation now instead. Much easier and less cables which pleases my wife grately!

The Dell SC440 was originally on loan from my brother and so I was concerned about not messing with it too much. He's since told me that if I want it I can keep it and do what I like to it, he's got better stuff now. Unlike your SC440s mine has a non-VT enabled processor which makes running ESXi 5 on it an impossibility.

This reminds me, no one has made much of a comment on the possibility of upgrading the processor in my Dell server. Did I scare everyone off with my concerns over destroying the computer? Did everyone else come to the same conclusion that upgrading it would be a waste of time, effort, and money? It seems that dales here has come to that conclusion. Unlike dales I don't have a wife to complain about the cables.

I'm thinking I could upgrade the memory a bit and use it to host some 32 bit VMs for use as servers for my lab. Not sure if it's a good idea, just tossing it out there. This Dell SC440 has an onboard gigabit NIC and I shoved two 100Mbit NICs in it's PCI slots. It's got two empty PCIe slots I could use for something. It also has a couple unused SATA ports and an empty drive bay or two. It's got some potential but I'm not sure I'd want to invest too much in a computer that is already around five years old.

Right now all I really use it for is web browsing and as a terminal for my Cisco gear, it could do so much more. That reminds me, I told myself I was going to study for my ICND2 test today. I got side tracked here. Back to my studies.

MentholMoose

MacGuffin wrote: »

You mention the value in having RAM. I have not experimented with ESXi 5 outside of a classroom so I'm still fuzzy on what I really need. Perhaps I've been placing too much focus on the processor and not enough on the RAM. How much RAM do I need? Or rather, with a given amount of RAM what can I expect from that computer? If I had to choose between gigahertz and gigabytes then what kind of calculus should I perform to determine the relative value?

In my experience, CPU performance is virtually irrelevant for labbing for the VCP and many other certifications. Look at the example of my Core 2 Duo based desktop... this CPU is only dual core, slower than and several generations behind the latest and greatest CPUs, and yet it works great for running a vSphere lab in VMware Workstation. The only problem is the lack of the CPU feature required to run nested 64-bit VMs (which, for the VCP, mainly means you cannot lab VDR, but otherwise is not a problem).

A low-end AMD CPU is perfectly adequate for lab purposes, as the example of my E-350 desktop shows... it is pretty much the cheapest AMD CPU (the CPU/mobo combo can be found for well under $100) and yet it can run an ESXi 5.0 VM in Workstation with nested 64-bit VMs. I'm sure the only thing holding it back is the lack of RAM and the host OS. Install 64-bit Windows or Linux and 8 GB RAM and I think it would be a nice lab machine on a low budget (sub $300 range).

The RAM is my main concern with the Mac Mini... a desktop with only 4 GB RAM (and maybe only a dual core CPU, depending on which i5 it is) and a regular hard drive (non-SSD) is really inadequate for a VCP lab machine, and for $700 you can do better (as Akaricloud showed). If you go the vSphere in a box route, with a 4 GB machine (regardless of vendor) you will only be able to run a single ESXi 5.0 VM (ESXi 5.0 won't install with less than 2 GB RAM) unless you upgrade to 8 GB. Worse yet, if 8 GB turns out to be inadequate, upgrading to 16 GB RAM is expensive (macsales.com has it for a whopping $235). I think 8 GB would probably be enough, though, so if you are okay with the cost of a Mac Mini with 8 GB RAM, it should suit your labbing needs.

MacGuffin wrote: »

This is the Nth time I've seen someone call for the use of SSD drives for the boot drive. I'm still not convinced this is as critical for performance as people claim. I'll do some reading on this later.

For a boot drive an SSD won't help that much. Boot time for the OS may be cut in half but it only saves seconds. Where it can really help, though, is with running VMs. An SSD may boot a single VM twice as fast as a regular hard disk, but that SSD can also boot 10 or more VMs simultaneously without any trouble, something that probably wouldn't work well with a regular hard disk. The VMs running in Workstation on my Windows 7 desktop (with its slow, outdated Core 2 Duo CPU) are stored on an SSD and run great... installing two ESXi VMs from an ISO simultaneously took 10-15 minutes, and installing the Fedora 16 VM from an ISO while also importing a Server 2003 template took about the same. Installing vCenter on that Server 2003 VM took 10 minutes, so I had a vSphere environment up and running in well under an hour.

MacGuffin wrote: »

This reminds me, no one has made much of a comment on the possibility of upgrading the processor in my Dell server. Did I scare everyone off with my concerns over destroying the computer? Did everyone else come to the same conclusion that upgrading it would be a waste of time, effort, and money?

What do you want to do with this machine, and what CPU are you considering for it? It looks like the SC440 uses an LGA775 CPU, but I don't think any LGA775 CPU supports Intel EPT. This is the required feature for running nested 64-bit VMs (Intel VT-x is not enough, you need VT-x with EPT, which I believe was introduced in the Nehalem architecture) so a vSphere in a box doesn't seem possible on that SC440 even with a CPU upgrade.

dales

It sounds like mentholmoose has a similar set up to me my host boot volume is just a standard 500GB sata disk which I've stuck a few iso's into as well, but I load my vm's onto 2 120GB ssd's and whilst they are still fairly pricey for what you get the difference is fantasic. I can boot esxi in under 10 seconds I can run 4 esxi instances and vcenter from the same ssd easily and there are no issues with with boot storming or general performance issue caused by too many IO's. my old sc440's have sata disks in them and running more than a couple of esxi instances caused noticable disk access issues depending on what I was doing with them.

CPU as MM said is not to be worried about, you should stick as much memory as you can afford in the box and also SSD's are a must if you want snappy vm's (also have a look at vinf's vtardis project for ideas).

I'm pretty sure I got 64bit vms to run nested on my sc440's although as I dont have them anymore I can't spin one up and try, my sc440's did have xeons in them though.

jibbajabba

I went for a Microserver from HP (which HP gives you cashback for) - put in an eBayed raid card and some cheap disks. Also upgraded to 8GB of RAM and added an IPMI card (which isn't necessary) .. Cheap and cheersful and all you need for labbing .... As Menthol said - CPU isn't the issue (mine runs at under 10% most of the time)

MacGuffin

MentholMoose wrote: »

In my experience, CPU performance is virtually irrelevant for labbing for the VCP and many other certifications. Look at the example of my Core 2 Duo based desktop... this CPU is only dual core, slower than and several generations behind the latest and greatest CPUs, and yet it works great for running a vSphere lab in VMware Workstation. The only problem is the lack of the CPU feature required to run nested 64-bit VMs (which, for the VCP, mainly means you cannot lab VDR, but otherwise is not a problem).

That clears things up. If I'm going to get a computer specifically for vSphere labs the ability for nested 64 bit VMs is a must, I'm willing to spend a few more dollars if I need to for this feature. I do not intend to host the virtual machines inside VMware Workstation if I can help it. I'd rather install the hypervisor on the metal to avoid the additional RAM requirements.

I feel I should point out that I need to be able to repurpose this computer in the future. In the next year I plan to get a CCNP and MCTS certification in addition to my VCP5. Being able to run the labs for those will be important as well. I doubt this will change my hardware requirements much, if any, but I need to keep that in mind as well.

MentholMoose wrote: »

A low-end AMD CPU is perfectly adequate for lab purposes, as the example of my E-350 desktop shows... it is pretty much the cheapest AMD CPU (the CPU/mobo combo can be found for well under $100) and yet it can run an ESXi 5.0 VM in Workstation with nested 64-bit VMs. I'm sure the only thing holding it back is the lack of RAM and the host OS. Install 64-bit Windows or Linux and 8 GB RAM and I think it would be a nice lab machine on a low budget (sub $300 range).

Outside of the feature limited Microsoft Hyper-V Server there is no 64-bit Windows OS for free. If I buy used or build my own then I'd have to add the cost of a Windows license and the VMware Workstation license to the cost. This is another reason I'm leaning toward a new computer from Apple, Dell, or whomever, the computer would come with a current and valid OS in the price. I don't need a VMware license since VirtualBox will do and is free. I also have the option of installing the hypervisor on the metal, improving performance, or downloading a 64-bit Linux to host the emulator.

Point is that I'm not seeing the cost savings claimed. I'm getting there. As I said before, it's been a long time since I had to look at computer hardware in such detail.

MentholMoose wrote: »

The RAM is my main concern with the Mac Mini... a desktop with only 4 GB RAM (and maybe only a dual core CPU, depending on which i5 it is) and a regular hard drive (non-SSD) is really inadequate for a VCP lab machine, and for $700 you can do better (as Akaricloud showed). If you go the vSphere in a box route, with a 4 GB machine (regardless of vendor) you will only be able to run a single ESXi 5.0 VM (ESXi 5.0 won't install with less than 2 GB RAM) unless you upgrade to 8 GB. Worse yet, if 8 GB turns out to be inadequate, upgrading to 16 GB RAM is expensive (macsales.com has it for a whopping $235). I think 8 GB would probably be enough, though, so if you are okay with the cost of a Mac Mini with 8 GB RAM, it should suit your labbing needs.

Again, I'm not finding these great deals that are claimed. Perhaps I've been dismissing the dual core PCs too quickly.

Yes, the 16GB RAM upgrade for the Mac Mini is expensive. A 16GB RAM upgrade for *any* PC is expensive. There's nothing special about RAM for the Mac Mini, it uses the same RAM as the Optiplex I was looking at. The 8GB chips can cost four times that of the 4GB chips. Unless I get a server motherboard (driving up costs) I'm going to be limited to motherboards with two or four RAM slots. I might just have to live with 4GB or 8GB RAM to keep costs low.

MentholMoose wrote: »

For a boot drive an SSD won't help that much. Boot time for the OS may be cut in half but it only saves seconds. Where it can really help, though, is with running VMs. An SSD may boot a single VM twice as fast as a regular hard disk, but that SSD can also boot 10 or more VMs simultaneously without any trouble, something that probably wouldn't work well with a regular hard disk. The VMs running in Workstation on my Windows 7 desktop (with its slow, outdated Core 2 Duo CPU) are stored on an SSD and run great... installing two ESXi VMs from an ISO simultaneously took 10-15 minutes, and installing the Fedora 16 VM from an ISO while also importing a Server 2003 template took about the same. Installing vCenter on that Server 2003 VM took 10 minutes, so I had a vSphere environment up and running in well under an hour.

I "misspoke" before. (mistyped?) I meant the drive that the VMs boot from, not the host. I should have called it the VM data drive or something.

I'll have to think about this. A SSD drive looks to be a cheaper way to boost performance than a faster processor.

MentholMoose wrote: »

What do you want to do with this machine, and what CPU are you considering for it? It looks like the SC440 uses an LGA775 CPU, but I don't think any LGA775 CPU supports Intel EPT. This is the required feature for running nested 64-bit VMs (Intel VT-x is not enough, you need VT-x with EPT, which I believe was introduced in the Nehalem architecture) so a vSphere in a box doesn't seem possible on that SC440 even with a CPU upgrade.

I'm not sure what I can use it for. I just gave a few possibilities that came to mind.

If I cannot get a processor that can host 64 bit VMs for it then I'm thinking I should just leave the processor alone. I'd consider some RAM and/or hard drive upgrade so it can make a good file server, host for 32 bit VMs, or whatever. A processor upgrade seems risky and has a low cost/benefit ratio compared to other options.

MacGuffin

dales wrote: »

It sounds like mentholmoose has a similar set up to me my host boot volume is just a standard 500GB sata disk which I've stuck a few iso's into as well, but I load my vm's onto 2 120GB ssd's and whilst they are still fairly pricey for what you get the difference is fantasic. I can boot esxi in under 10 seconds I can run 4 esxi instances and vcenter from the same ssd easily and there are no issues with with boot storming or general performance issue caused by too many IO's. my old sc440's have sata disks in them and running more than a couple of esxi instances caused noticable disk access issues depending on what I was doing with them.

CPU as MM said is not to be worried about, you should stick as much memory as you can afford in the box and also SSD's are a must if you want snappy vm's (also have a look at vinf's vtardis project for ideas).

I'm pretty sure I got 64bit vms to run nested on my sc440's although as I dont have them anymore I can't spin one up and try, my sc440's did have xeons in them though.

No Xeons here, I got a Pentium D.

I did a quick look at some SSD drives. Are the 6GB/s drives worth the cost difference over the 3GB/s drives?

Looks like the 60GB SSD drives have the best bang/buck ratio. That should be enough if I don't go overboard on the VMs and have a spinning metal drive for less critical data.

MacGuffin

jibbajabba wrote: »

I went for a Microserver from HP (which HP gives you cashback for) - put in an eBayed raid card and some cheap disks. Also upgraded to 8GB of RAM and added an IPMI card (which isn't necessary) .. Cheap and cheersful and all you need for labbing .... As Menthol said - CPU isn't the issue (mine runs at under 10% most of the time)

Cashback? You mean they will buy them back from you when you're done with them?

I'll take a look at the HP Microserver.

MentholMoose

MacGuffin wrote: »

That clears things up. If I'm going to get a computer specifically for vSphere labs the ability for nested 64 bit VMs is a must, I'm willing to spend a few more dollars if I need to for this feature.

My point was that you don't need to spend extra for a high-end CPU for labbing. You just need one that supports the required features. All that should be necessary is a recent CPU (released in the last two years or so) that supports VT-x or AMD-V.

MacGuffin wrote: »

I'd rather install the hypervisor on the metal to avoid the additional RAM requirements.

There is not that much difference in memory overhead at this scale. Running ESXi directly on the hardware will only save a few hundred MB, not enough to run additional ESXi VMs (ESXi won't even install with less than 2 GB RAM). ESXi can overcommit RAM so you could possibly boot two ESXi VMs on a 4 GB machine, but once you do anything in those VMs it probably wouldn't work well.

Using Hyper-V on Windows Server or Workstation/VirtualBox on Windows 7 will be about the same since either way there is a Windows OS eating RAM. Using Hyper-V Server or Hyper-V on Server Core would help somewhat since there is less stuff installed to waste RAM, but it would be similar to ESXi and only save a few hundred MB. Basically, 4 GB RAM is quite limiting, regardless of the hypervisor, so I recommend 8 GB RAM at minimum.

MacGuffin wrote: »

Point is that I'm not seeing the cost savings claimed. I'm getting there. As I said before, it's been a long time since I had to look at computer hardware in such detail.

I'm more familiar with DIY configurations and deals than the system deals you can get from Dell, HP, and the like. To give you an idea of how cheap a lab machine can be, here's a low-budget bare-bones configuration (all prices from Newegg) that requires minimal assembly (only install RAM and a disk, the rest is pre-assembled).

• Bare-bones desktop with AMD E-350 CPU - $130 - Foxconn SFF R20-A1 (it was on sale for $95 a few weeks back)
• 8 GB RAM - $35 - cheapest 8 GB (2 x 4GB) DDR3 kit
• 60 GB SSD - $75 - OCZ Vertex 3 60 GB
• Linux host OS - $0
• VMware Workstation (free for me as I work for a VMware partner) or VirtualBox (also free)

While being budget-oriented, the included E-350 CPU is fairly new and supports AMD-V with RVI (the equivalent of Intel VT-x with EPT). I have a similar machine (mainly used by my wife for email and web browsing ) and nested 64-bit VMs work fine in ESXi VMs running in Workstation. ESXi may or may not install directly on it but for $240 (or less if you find a promotion or deal) it is hard to beat for a vSphere in a box lab. Of course if you consider requirements besides labbing, the Mac Mini may still end up being the right option, but for labbing the VCP (and many other certs) I'd take the Foxconn bare-bones system any day.

MacGuffin wrote: »

Again, I'm not finding these great deals that are claimed. Perhaps I've been dismissing the dual core PCs too quickly.

A quad-core CPU is nice but not essential for labbing. Are you sure the Mac Mini you mentioned is not dual-core? What is the exact Core i5 CPU in the Mac Mini you mentioned? Most i5's are dual-core so if you want quad-core you should check that.

MacGuffin wrote: »

Yes, the 16GB RAM upgrade for the Mac Mini is expensive. A 16GB RAM upgrade for *any* PC is expensive. There's nothing special about RAM for the Mac Mini, it uses the same RAM as the Optiplex I was looking at. The 8GB chips can cost four times that of the 4GB chips. Unless I get a server motherboard (driving up costs) I'm going to be limited to motherboards with two or four RAM slots. I might just have to live with 4GB or 8GB RAM to keep costs low.

For standard desktops that take regular DIMMs (and not SO-DIMMs), it is not very expensive at all. A four DIMM 16 GB DDR3 kit is $70 from Newegg, whereas a two DIMM kit is $105.

Since you mentioned a server, I happened to have recently built one. The total cost was about $800. Here's the config:

• Supermicro H8SCM-F mobo - $210 (new, includes IPMI so I can remotely manage the power state, access the console remotely, reinstall the OS remotely, among other things)
• Kingston 32 GB Registered ECC DDR3 - $260 (new, 4 x 8 GB DIMMs)
• AMD Opteron 4162 EE CPU - $100 (hex-core, low voltage, "like new" on eBay)
• Case / PSU / HSF - $0 (salvaged from an old machine, otherwise $60-80 for a mATX case/PSU combo and an HSF)
• 2 x Sandisk 120 GB SSDs - $230 (new)

This is relatively recent server-grade hardware so as you might expect, it works very well with ESXi. ESXi 5.0 update 1 installed without any hassle (no missing drivers, etc.) and performance is superb. To recap, for $100 more than that $700 refurb Mac Mini you mentioned, I got a (mostly) brand new server with IPMI, a hex-core server-grade CPU, eight times the RAM (and it's server-grade with ECC), and 240 GB of SSD space. At least for my needs, each of these are extremely significant advantages, well worth the slight premium.

MacGuffin

My point was that you don't need to spend extra for a high-end CPU for labbing. You just need one that supports the required features. All that should be necessary is a recent CPU (released in the last two years or so) that supports VT-x or AMD-V.

I'll half-way agree with you here. There's still plenty of new and refurbished computers out there with Core 2 Duos. The Core 2 Duos are cheaper than the i3/i5/i7 computers. That means that if I want to do nested 64 bit VMs (which in the case of a lab-in-a-box is a necessity) I'll need to spend a bit more money to get an i5 or i7.

Using Hyper-V on Windows Server or Workstation/VirtualBox on Windows 7 will be about the same since either way there is a Windows OS eating RAM. Using Hyper-V Server or Hyper-V on Server Core would help somewhat since there is less stuff installed to waste RAM, but it would be similar to ESXi and only save a few hundred MB. Basically, 4 GB RAM is quite limiting, regardless of the hypervisor, so I recommend 8 GB RAM at minimum.

I get your point but I'm not sure you got mine. I would think that a few hundred MB would be kind of a big deal on a box with only 4GB total. I agree though that 8GB is a minimum. The price to upgrade just about any box I've looked at so far to 8GB would be less than $50, and that makes it money well spent.

Of course if you consider requirements besides labbing, the Mac Mini may still end up being the right option, but for labbing the VCP (and many other certs) I'd take the Foxconn bare-bones system any day.

I like the Mac Mini for many reasons. It's a small, compact, quiet, and energy efficient box. It's a known quantity to me since I've been using Apples daily since I was in fourth grade, and I've been treated well by Apple's customer support when I needed it (which was rare). To be fair I also have plenty of experience with Dell gear having used Dells for many years. They have proven to be reliable as well. There's a certain comfort in knowing that what I buy will be reliable and that I can get it fixed if I must. That is worth something to me, I'm just not sure how that relates to dollars. Stepping out of that comfort zone and into a brand I do not know is a cost that I'm not sure how to price.

A quad-core CPU is nice but not essential for labbing. Are you sure the Mac Mini you mentioned is not dual-core? What is the exact Core i5 CPU in the Mac Mini you mentioned? Most i5's are dual-core so if you want quad-core you should check that.

I believe I've mentioned four different Mac Minis in this thread, the cheapest one was a Core 2 Duo, the most expensive one was a quad Core i7. I just checked the Apple store web site and they no longer list the quad core Mac Minis on the refurbished gear page. That means the price just went up about $150 for those since I'd have to buy new now instead. The Core 2 Duo Mac Minis just dropped about $50 though. I'm not sure how many cores were in the i5 Mac Minis I mentioned before, it's moot now since they're not for sale any more. All the new Mac Minis with i5 processors have two cores, and they cost more.

From what you wrote and from other comments I've read I'm convinced I don't *need* quad core for VMware but I've had Cisco on my mind the last couple days and it appears that a quad core is nearly a necessity for GNS3 labs. I'd prefer quad cores so that I can move this computer into a Cisco lab-in-a-box role as well.

This is relatively recent server-grade hardware so as you might expect, it works very well with ESXi. ESXi 5.0 update 1 installed without any hassle (no missing drivers, etc.) and performance is superb. To recap, for $100 more than that $700 refurb Mac Mini you mentioned, I got a (mostly) brand new server with IPMI, a hex-core server-grade CPU, eight times the RAM (and it's server-grade with ECC), and 240 GB of SSD space. At least for my needs, each of these are extremely significant advantages, well worth the slight premium.

I guess I'm not as impressed with "server grade" as much as I used to. I remember when that was a big deal back when computers were much more expensive and not as reliable as they are now. I had to look up what IPMI was and I doubt I'd ever use it. I could do without the dual drives, this isn't going to be a file server, one should be enough. What does impress me is how cheap the memory is compared to the memory I saw for the Mac Minis and the SFF Dells I was looking at. I don't know how a hex core AMD compares to a quad core Intel. There might be a significant gain there, but it might also be a loss. I'll have to look into that.

As I mentioned before there is a certain value in dealing with the familiar. I know what a Mac Mini can do and I know how to fix Apples if something goes wrong. There's all kinds of information on how to get ESXi on Mac Minis and outside of an easily fixed NIC driver issue it appears the Mac Mini has served others well as a host for ESXi.

You gave me some good stuff to ponder. I'm going to try to push off this purchase as long as I can to see if a good deal and/or more funds come along. I've decided I'm going to do some studying for my Cisco exams for now. I'll see how long I can go without buying more hardware and keep doing some research on this once in a while.

MentholMoose

MacGuffin wrote: »

I like the Mac Mini for many reasons. It's a small, compact, quiet, and energy efficient box.

I care about these qualities as well (energy efficiency is especially critical). Both of the systems I spec'd out meet them, but they are obviously not quite as mini as a Mac Mini.

MacGuffin wrote: »

It's a known quantity to me since I've been using Apples daily since I was in fourth grade, and I've been treated well by Apple's customer support when I needed it (which was rare).

I tend to go with brands I know or can research. I've been building systems for over 10 years so I am familiar with a variety of brands and have my favorites.

MacGuffin wrote: »

There's a certain comfort in knowing that what I buy will be reliable and that I can get it fixed if I must. That is worth something to me, I'm just not sure how that relates to dollars. Stepping out of that comfort zone and into a brand I do not know is a cost that I'm not sure how to price.

One reason why I like DIY systems is that the warranty terms are usually lengthy for components... typically 3-5 years, and in some cases (like with most RAM) lifetime. I don't know about Apple but other system vendors I've dealt with charge you extra to have more than 1 year of support. Buying from a system vendors has some benefits, though, such as the convenience and consistency of support you get with the entire box being supported by one vendor (as long as that vendor has good support, of course).

MacGuffin wrote: »

I guess I'm not as impressed with "server grade" as much as I used to. I remember when that was a big deal back when computers were much more expensive and not as reliable as they are now.

The quality is not my concern. A desktop from a reputable vendor will have similar quality to a server from a reputable vendor. To me the main benefit of a server is compatibility. I can't say for certain but I doubt the designers of the Mac Mini cared if ESXi worked well or even at all on it. Supermicro, on the other hand, know that people are running server and virtualization workloads on their servers, so they design and test them accordingly. The vSphere HCL shows this well... Supermicro has over 100 systems on it, whereas Apple has ONE (the discontinued Xserve).

I have read some articles and forum posts about installing ESXi on a Mac Mini so I've heard of the problems like the keyboard not working and NICs not being supported. People have figured out how to resolve these (and have posted articles with the solutions), but I just don't want to deal with these or other hassles. I want it to just work, so a Supermicro server is kind of like my "known quantity" solution.

The second most important benefit is feature-set. You can't really get out-of-band management on a desktop unless you spend a fortune on an IP-based KVM, and this feature was necessary for the Supermicro system I built. The pricing seals the deal. The Mac Mini is just not remotely competitive price-wise for my needs.

MacGuffin wrote: »

I could do without the dual drives, this isn't going to be a file server, one should be enough.

The option to have multiple drives is important to me since I use SSDs. If I buy, say, a 60 GB SSD, and two months down the line I find that 60 GB is not enough, I don't want to have to replace it with a bigger SSD. It saves time and money to just install another SSD. Multiple drive support is also useful since you can install a cheap, big regular hard disk for certain data (hypervisor installation, ISOs, VM templates, etc.) and an expensive, smaller but much faster SSD for VMs.

jibbajabba

Re Licenses .. if you intend to do a lot of labbing in the future - especially MCTS (which effectively requires Windows licenses one way or another) have a look at the single user Technet sub ...

Edit: Oh and I love Supermicro - rock solid .. never looked back (all my server run Supermicro in both flavours, superserver and DIY)

MacGuffin

jibbajabba wrote: »

Re Licenses .. if you intend to do a lot of labbing in the future - especially MCTS (which effectively requires Windows licenses one way or another) have a look at the single user Technet sub ...

Edit: Oh and I love Supermicro - rock solid .. never looked back (all my server run Supermicro in both flavours, superserver and DIY)

I'll have a look at Technet. Since this is a forum on virtualization I'll try not to step too far from the issues of getting an ESXi lab running. I realized that I will have to think a bit about the software costs.

With the hardware lists that MentolMoose has given the idea of building my own PC for this project does look to be a bit appealing. I'm still not sure I want to bother with the hassles of piecing a computer together from parts but with proper planning it might lead to significant savings. What makes the purchase of a pre-assembled PC appealing is that there is no assembly required, there's only one phone number to call if there is a hardware issue, and there is an operating system included in the price with all the needed drivers included as well. MentholMoose even admits to this after pointing out the cost savings of a DIY box. It seems that no matter what hardware I choose, excepting a server on the VMware HCL (which will be expensive), I run the risk of some sort of driver/compatibility issue.

MentholMoose wrote:

I have read some articles and forum posts about installing ESXi on a Mac Mini so I've heard of the problems like the keyboard not working and NICs not being supported. People have figured out how to resolve these (and have posted articles with the solutions), but I just don't want to deal with these or other hassles. I want it to just work, so a Supermicro server is kind of like my "known quantity" solution.

I have read this articles as well. The keyboard issue is well known, has numerous workarounds, and affects only older iMac models (from 2009 and 2010 IIRC). In my research on the Dell systems the workarounds I've found the incompatible hardware issues are resolved in much the same way as the Mac Mini. The difference is that with the workaround for the Dell systems the most prominent resolution was to buy more hardware. I'm not sure what a SATA controller or PCIe NIC costs but for a price conscious person like myself there is a desire to avoid having to buy more hardware to solve a software issue.

It seems to me that these driver issues would become moot if I do as MentholMoose has suggested and run ESXi inside another virtualization product. This is where the software costs come in. If I am to properly evaluate the costs of the different options before me then I need to add in the costs of the software as well. I've been focusing on running ESXi as a host for my lab-in-a-box primarily because the software is free. Free is good but as we have touched on many times already in this thread there is a good chance I might not be able to get ESXi 5 installed on the metal and still have access to vital hardware like the keyboard or NIC.

Linux is free. VirtualBox is free. VirtualPC is free with the purchase of Windows 7, but Windows 7 is not free. VMware Player is free, but feature limited. VMware Workstation has a try before you buy evaluation period. Windows Server 2008 has a try before you buy evaluation period as well, which includes the Hyper-V hypervisor. A feature limited, but free, Microsoft hypervisor OS is available as well. Any new computer system will come with an OS in the price. Some used systems might have a suitable OS as well. A DIY system will obviously not come with any software.

The problem I'm having in evaluating the software costs is the lack of information. There is not much information out there for situations outside of hosting ESXi on another VMware product. I did find in my research that VirtualBox cannot host ESXi, something that my own testing has confirmed. My Google searches have not turned up much information beyond that.

What I have found is that if I end up with some sort of driver/compatibility issue with the free options I could easy end up spending $400 on software to resolve the issue. I found VMware Workstation 8 for $199, full version of Windows 7 Home for $199, more feature rich versions of Windows cost more of course. Just the cost of the OS makes the DIY option look less attractive. Since I do not plan to just toss this computer in the dustbin after using it for my VCP studies I'm going to want some sort of OS on it at some point anyway. The OS does not have to be Windows but that might end up being preferred in the end for a variety of reasons.

Anyway, I've likely babbled on long enough to make my point. Some guidance on the software aspect would be appreciated. I did happen across a discounted bundle of software that included VMware Fusion 4 so that lowers the cost of entry for the Apple solutions a bit. I bought the software and played with it for a bit today on my MacBook Pro. I was able to get ESXi 5 to install. To get any further I believe I'll need to close all other programs on my laptop to free up more RAM and CPU cycles. This is only to prove the concept of hosting ESXi on *A* Macintosh, not *THIS* Macintosh. The time it took to install the hypervisor has already shown this computer is not up to the task of hosting even one instance of ESXi.