Hardware selection help

dfortier · February 2014

Hi all, I just stumbled on this site yesterday and have been doing a fair amount of readying since. I'v recently decided to pursue (after 20+ years) certifications for myself (and hopefully secure my future). My main area of focus is virtualization for two an reasons. First my current employer is starting to see the benefits (and will pay some costs) and second I just love working with it. As part of my "re-education" I will be taking a number of other basic exams as well (A+, network+, Linux+).

With the ultimate goal being VMCP or better, im looking for a bit of advice on a hardware front. I used a home built AMD8 core "server" with 32GB ram and 4 1TB raid 10 and 3TB storage drive for VCA-DCV (passed on Friday, go me). issue im running into more than anything else is ram or lack there of rather. Between home machine (minecraft, media server, backup), VCenter, Unitrend backup (great freebie) and a few test VMs im out rather quickly. So i'm thinking a used amd server may be the answer.

I'm looking for he following:

quiet as possible (this is the big one, i can look up the specs myself if need be, noise is hard to just in review videos, my have to build my own?)
preferably up to 128GB or better ram
Dual cpu capable or at least 12 core + if single
allow for nesting so i can play with HA/FT (old system may become isci san or second host)

thanks for he input

SimonD. · February 2014

First of all I would ask the spec's of the machines you already have running because 32gb and you're struggling sounds like you have over provisioned your VM's.

I can run a domain controller, sql db server, vcenter server and media server using 2gb, 6gb, 4gb and 1gb respectively, that doesn't mean however that all of that ram is actually being utilised, as it currently stands I am actually only using about 7gb of ram at the moment.

First thing I would look at is reducing your allocated ram per VM, I would also look at how many vcpu you are allocating (more does not equal best in this instance).

Now on to your server, you may well be better off building a white box rather than getting boxes with that amount of ram (ECC ram fitted to servers is not cheap, you're looking at over $1000 for 128gb of ram on ebay at the moment, that excludes the server hardware).

I built up a 96gb cluster of servers for about $2000, these are three 32gb Shuttle SH67H3 servers with 4port Nics and Intel I7 3770 cpus fitted, sure they aren't dedicated servers but they run all the infrastructure I have thrown at it (vSphere, vCloud etc). It also allows you to scale up when you're ready.

Have a look at my lab details on my blog below.

Essendon · February 2014

Yeah in addition to what Simon's said - your bottleneck may well be disk. How do you know your struggling for RAM? Is it you cannot build more VM's or are the VM's demanding more? So whittle down the amount of RAM your VM's have. For example, I have been using a 2008 R2 domain controller for ages with 512MB RAM, with NO trouble.

128GB RAM?? Jeez, are you planning to launch a geo-spatial orbit satellite?? I got by with 8GB RAM for my VCP5!! Sure the machine wasnt blazing fast, but it did the job. For my VCAP's, I have used a DL 380 G6 with 48GB RAM and my setup's nested too and the thing's a beast - doesnt miss a beat. You should reconsider your options. Let us know if you need more info.

dfortier · February 2014

Thanks for the advice Simon. Right now i'm using the following:

sys cpus ram
minecraft 1 4GB
VC-App 2 4GB
Plex 2 8GB
UniTrend 2 4GB
DC1(2008r2) 1 4GB

Nest ESX1 2 8GB
Nest ESX2 2 8GB
Starwind 2 8GB

The last 3 are for learning, but the others are "got to have them". I thought about building another desktop to separate the house needs from lab, but I thought a used server might be cheaper and more expandable. I'll take a look at your setup

thanks

dfortier · February 2014

Essendon wrote: »

Yeah in addition to what Simon's said - your bottleneck may well be disk. How do you know your struggling for RAM? Is it you cannot build more VM's or are the VM's demanding more? So whittle down the amount of RAM your VM's have. For example, I have been using a 2008 R2 domain controller for ages with 512MB RAM, with NO trouble.

128GB RAM?? Jeez, are you planning to launch a geo-spatial orbit satellite?? I got by with 8GB RAM for my VCP5!! Sure the machine wasnt blazing fast, but it did the job. For my VCAP's, I have used a DL 380 G6 with 48GB RAM and my setup's nested too and the thing's a beast - doesnt miss a beat. You should reconsider your options. Let us know if you need more info.

I keep getting alarms that ram usage is high, and things seem really slow. I actually down loaded VM-IO-Analyzer this morning but haven't put it on yet.

Essendon · February 2014

There you go, your lab's overprovisioned.

- DC1 -> does it do anymore than directory services? If not, pare it down to 1GB RAM.
- Nested ESXi instances -> pare them down to 6GB RAM. They dont need 8GB RAM in a lab, in most cases.
- Starwind -> 512MB RAM's enough. I run Starwind too and have vCenter installed on the same VM and I have only given the VM 4GB RAM, no problems so far I have SRM installed on another machine too and have no problems with replication and all.

Do the above and you've already got like 14GB RAM spare!

dfortier · February 2014

thanks for the advice, ill give it a shot.

msteinhilber · February 2014

Yea, as already pointed out - you're overprovisioning. I have an i7-920 with 24gb of RAM and I ran into disk I/O issues and never have any memory issues. You're running a lab, not a production environment so you can skimp a bit. I/O became a bigger problem for me, so I just ended up taking my VM's off my RAID5 array with 3 1tb WD Blacks and put them on two 240gb SSD's. I run a few VM's that I keep doing that aren't part of my lab as well with this setup and it's fine. Not as many as you do but I run a Minecraft server for the kid, a Win7 VM that my IP cameras use as their DVR, Plex to stream to my Roku box and a small Syslog server. My lab could be quicker, sure, but it's not slow enough for me to warrant a 2nd rig to lab on and the power costs associate with it... my energy bill is high enough already!

dfortier · February 2014

Just wanted to post back with a thank you. I slimmed down some VMs and moved them over to a SSD as suggested and thing seem a lot smoother now.

kriscamaro68 · February 2014

If you are still needing more then get one of these: HP Proliant DL160 G6 1U 2X Xeon QC L5520 2 26GHz No HDD 72GB RAM Special Qty 000491532004 | eBay

I bought 2 of them for my studying. I like using the physical hardware for my cluster setups. If you bought 1 though you could easily run a nested setup on it as it has 2 quad core's with hyper-threading. I have a single 120gb ssd in each server for the OS and then connect to my 2012 storage server for my cluster storage. Works quite well. Each server uses about 90 watts on a regular basis. (tested with a killawatt) They are actually pretty quite to after they are done booting. If you can put them in a basement or wire closet or something you would never hear them.

dfortier · February 2014

thanks for the input, if I find I still need a little more I may go that route.

Hardware selection help

Comments