Building an energy-efficient ESXi Server

Con7e

Hello All, I'm new around here.

I just passed the CCIE R&S Written and I am looking to build a home-made lab to practice for the CCIE.

My intention is to replicate INE's topology (but without the physical switches, as for those I have already tickets for their racks).

Having a budget of 1.5 / 2K, what system could i build to not run into any issues ESXi-wise? On google the first result brings up this: Building Energy Efficient ESXi Home Lab - parts list - ESX Virtualization

That post does seem to somewhat focus on the drives (of which I don't care much about considering for what I will be needing this system).

Are there maybe ready-to-buy systems where I can Just thourgh in 80 GB of ram and be done with it?

Any help would be appreciate!

Thanks.

gumpware

Here's another option, courtesy of the RouterGods:

RouterGods-Validated Shuttle Build

Con7e

Hi gumpware,

I actually went with the following: Supermicro SYS-5028D-TN4T-BUNDLE3BV Embedded Processor

Thanks for your suggestion though!

Bardlebee

gumpware wrote: »

Here's another option, courtesy of the RouterGods:

RouterGods-Validated Shuttle Build

Can confirm. I have this very build. Finally finished my setup for both VIRL+CSR's and GNS3. Time consuming but worth the experience since I was fresh to the virtualization realm.

Con7e

Bardlebee, would you share some kind of schematic on how you setup your lab? Considering we are "in the same boat" I would really appreciate your input.

Bardlebee

Oh man, it was pretty involved. But I'll try to field any questions you may have.

As far as what I'm running. I subscribed to VMUG and downloaded the latest ESXi server. I installed it via USB and put it on my network. I then installed VCenter which helped me a lot. Both of these things I found tutorials on how to set this all up.

After you have those things in, you can download the CSR's and subscribe to VIRL if you want to leverage it as a layer 2 setup.

How to build CCIE v5 Lab with Cisco CSR 1000v Routers

I followed this generally for the CSR's, what you'll need to do is setup a vSwitch network (I just used my base network and changed it) and then you will want to make it switching, so basically have all the vlans on the switch/hub so that when you're tagging on one CSR, the other ones can see it. Provided they have the tags.



Note the VLAN tagging settings, put VIRL and the CSR's into this type of network



Now, you should be able to make sub-interfaces and as long as the CSR's are tagged the same on each side, reachability/pinging should be there. To bridge this same gap with VIRL as your layer 2, you'll want to setup your network topology as "Flat Network" this is a setting you'll see under VIRL if you have a topology and you just click the general background in the "Design" portion. Change the network type to "Flat"

You will also need to set your Promisucous Mode on your vSwitch. From there you will be able to trunk your g0/0 interface on your VIRL switch and reach out to the sub-interfaces of your CSR's. To test this, just make a VLAN and an SVI in the same space and attempt a ping.

Note, that the vSwitch is really doing all your Layer 2 here, but for the sake of INE and being able to setup neighborships with your switches between your CSR's, this will need to be in place to emulate that. What is really going on between your CSR's/VIRL is basically a hub and it just shares all vlans with all ports. The goal is to bridge the VIRL+CSR gap. Which Promiscuous mode fixes.



Let me know if you have any other questions. I'll try to help. It took me quite a long time to get that far so hopefully this cuts down on your time.

EDIT: I'm currently transitioning to Narbik's labs which I use GNS3 VM's for which is a whole other ball of wax but in general was an easier setup.

Con7e

Thanks a lot mate.

ITtech2010

Hey guys,

I currently have a hackintosh and running 32gb of ram and a i7. It's a pretty beefy machine with plenty of hard disk space. I am running El Capitan 10.11.5. I've been contemplating swapping out the processor (for skylake)/ram (DDR4 64-12/motherboard. For the CCIE lab training you definitely recommend between 64 to 128ram?

Con7e

I bought my server with 64 Gb, and that should be enough to run 20+ csr routers and some virtual machines.

ITtech2010

Are you running 3gb or 4gb per CSR? I think the requirement for each is 4gb and 1 processor. But you can probably get away with 3gb (just slower). If you run 20 csr at 4gb it will be about 80gb. at 3gb per CSR it will be about 61gb.

Con7e

If you look at the requirements on Cisco's website, each subsequent version of the CSR has a lower hardware requirement.

The version I plan on using (which runs n ESXi 5.0, 5.1 and 5.5) requires only 1 vCPU and 2.5 Gb of ram. That's 50 Gb of ram in total.

But consider this: The routers in reality use most of the resources only when they boot up. afterwords the RAM/CPU usage is a lot lower. ESXi has the capability to share memory between different virtual machines. Hence the actual memory requirements is lower than 50 Gb once every router is up and running.

In my opinion 128 GB are waay overkill.

ITtech2010

Con7e wrote: »

If you look at the requirements on Cisco's website, each subsequent version of the CSR has a lower hardware requirement.

The version I plan on using (which runs n ESXi 5.0, 5.1 and 5.5) requires only 1 vCPU and 2.5 Gb of ram. That's 50 Gb of ram in total.

But consider this: The routers in reality use most of the resources only when they boot up. afterwords the RAM/CPU usage is a lot lower. ESXi has the capability to share memory between different virtual machines. Hence the actual memory requirements is lower than 50 Gb once every router is up and running.

In my opinion 128 GB are waay overkill.

Wow I did not know that Con7e. That makes a lot of sense in this case. Maybe the reason the resources were overkill was because I am using the VIRL images. When I used it for GNS3 i know that the images were running slow directly off OS X. When I imported the image into GNS3 VM it worked a little better. I'm going to tinker with the resource in my lab at home and at work. (OS X and Windows 7/2012) I will see how it behaves after they boot up which will probably take a long time. If it's operational then your right 128 would be way overkill.

I also use F5/Fortigate/Arista appliances. I was considering mocking my whole data center design. That's why I was thinking 128 gb might be worth it though.

ITtech2010

Hi Con7e,

You were right. I throttled the ram down to 2.5gb and powered on two of them. It works and the cli runs ok. The only pain with using it in gns3 is that you can't connect the cable while powered on during the lab. And I have a direct connection between two CSR hosts and ping connectivity is not working. Might just be an issue on the windows version. But I was able to run it but it did take at least 25 minutes to load both CSRs.