VMware ESXi 4.0 deployment ideas for lab.

BroadcastStorm

Hi guys, so I was reading alot about VMware ESXi last night, and discovered that this is now a free software.

I've worked with vsphere/esxi before and wanted to virtualize my web server, as well openfiler, on an AMD X2 64 5600 processor, I have a quad core from my gaming rig, David Davis advised to continue with the install even if it's not on the HCL.

Anyway my install plans are the following:

Disk 0 - esxi hypervisor install
Raid 5 - Independent VMguests

I also plan virtualizing openfiler, haven't had a chance to read about openfiler best practice but this should get me started, please let me know your suggestions/ideas.

The plan is to get a FC switch and using my old DL380 as the storage with FC cards to play around SAN.


Thanks!

QHalo

Updated 2011 vSphere ESXi 4.1 (and future) Homebrew/Whitebox parts list. - Virtual Geek
http://www.techhead.co.uk/vmware-esxi-home-lab-why-what-and-how-considerations-when-building-your-own-home-lab
http://www.vm-help.com/esx40i/esx40_whitebox_HCL.php
http://ultimatewhitebox.com/systems

I would also look into using Uber VSA or even Starwind virtual devices as well.

You might have a look at this thread over on [H]ardForums which has a ton of information about what other people are doing.

Your home ESX server lab hardware specs? - [H]ard|Forum

hth

MSNinja

Hi, David Davis is right, just try setting it up. Just be sure you have a Intel or a Broadcom NIC.
If your BIOS/UEFI supports USB booting then you can boot ESXi from there. And use ESXi 4.1 U1 if you are setting this up, should be available for download as a free version already.

zerglings

Not sure if you're still looking at building a white box, but I just built one and maybe you're interested building a similar one as mine.

AMD X6 1090T with ASUS M4A88TD-V EVO/USB3 - you need to downgrade the BIOS to 1106
G.SKILL Ripjaws 8GB DDR3 1600
Intel Gigabit NIC
Corsair AX850 PSU
Perc 5i RAID Controller with SAS to SATA cables
Old 4GB USB flash drive - VMware ESXi 4.1
2 x WD Caviar Black 640GB
2 x WD Caviar Black 750GB
2 x WD Caviar Black 1.5TB

BroadcastStorm

zerglings wrote: »

Not sure if you're still looking at building a white box, but I just built one and maybe you're interested building a similar one as mine.

AMD X6 1090T with ASUS M4A88TD-V EVO/USB3 - you need to downgrade the BIOS to 1106
G.SKILL Ripjaws 8GB DDR3 1600
Intel Gigabit NIC
Corsair AX850 PSU
Perc 5i RAID Controller with SAS to SATA cables
Old 4GB USB flash drive - VMware ESXi 4.1
2 x WD Caviar Black 640GB
2 x WD Caviar Black 750GB
2 x WD Caviar Black 1.5TB

I installed VMware ESXi 4.1 on a Dell Poweredge 1800, I noticed that when I was provisioning a 64 Bit VMGuest it was complaining.

So I will be working on the following once I call the local VMware academy around my area to attend the class.

ESXi1 - IBM Rack @Server 325 Opteron 64-Bit - I will be designing HA in here etc.
ESXi2 - Asus Crosshair 3/AMD 64 Bit 4 Cores - I will be designing HA in here etc.
ESXi3 - Dell Poweredge 1800 Intel Xeon. - This is for my production stuff
Openfiler - Asus A8NSLI-Deluxe/AMD 64 Bit CPU - Independent SAN/NAS storage.

I would think that VMware HA would be supported on the 2 AMD build?

Then I will be connecting it to Openfiler that I recently configured.

Would it be necessary to peer with a buddy to test VMware cloud computing? I have a ASA 5505 if anyone is interested doing S2S VPN Tunnel, and play around VMware.

cyberguypr

Zerglings, what will the storage configuration be? All local? do you have only 1 ESX host?

zerglings

I apologize for a lengthy post but I decided to share my nightmare story just in case someone out there is reading this that requires the same setup as mine or similar.

ESXi weekend project turned into a nightmare which I decided to put on hold until there's a need for me to run two computers 24/7.

I've been running my old PC (P4 3.4GHz with 2GB RAM) 24/7 for quite sometime now. It is loaded with Win 2K3 as the host OS and running XP and Ubuntu Server as guest OS running as a service. XP wasn't part of the setup before because I was able to run MagicJack on 2K3. I mainly use this home server, if you will, as my DC, File server, FTP server, SSH server, crontab (I used it to schedule waking up my HTPC to record TV shows), and as a proxy server. Everything works great but for the past several months, the PC has been rebooting by itself. I knew that the problem was the PSU, but I decided not to buy just that since I am already limited on what I can run because of the CPU and amount of RAM installed. So, I decided to build a new one all together with a requirement of at least 80 Plus Bronze PSU and completely modular. Didn't find any that is reasonable so I decided to get a little crazy and gotten me an expensive PSU, Corsair AX850.

Anyway, what I wanted ESXi to do was to run the same thing as my old setup. However, I found out the hard way. I should've researched before jumping to conclusions. Here are the things I found out during my weekend project that made my week all messed up:

- I found out that my IDE DVD/RW was not compatible with ESXi.
- I found out that when you install the ESXi on Flash drive it will automatically format the attached drives on my Perc 5i with VMFS - will be relevant later.
- I found out while I was transferring all my files to VMFS formatted drives that the maximum throughput of ESXi is not that high, hovering around ~40 MB/s and I was transferring about 1TB of data.
- I found out that since ESXi formatted my RAID 1 1.5TB drive and RAID 1 640GB to the default block size of 1MB automatically, I was limited to only 256GB of data. I was scratching my head why I was getting this I/O occurred error. After reading several forums, people were suggesting to install FastSCP or WinSCP to transfer files. Did that, but finally the I/O occurred error that I was getting was because of no space on the drive
- I found out that when ESXi formatted my 1.5TB to 1MB block size, I couldn't take it out from storage section. Tried different things suggested on VMware forum and nothing worked except the guy who suggested to format it to other file system and ESXi should see it as a new drive.
- I found out that I messed up one of my drives that contained data which I had to recover on another machine by trying to initialize the drive on ESXi.
- I found out that my whitebox build does not support VMDirectPath so OpenFiler is out and MagicJack is out of the picture. MagicJack is not that important to me since I can always run it on my old laptop, but I was really trying to avoid having two PCs running 24/7.
- I found out that I didn't have enough patience to deal with manually copying folders that contain a lot of data and at a throughput of ~40MB/s. I'll probably still moving files around if I hadn't stopped.

Now, I am going back to a similar setup as my old one which I know works. I basically wasted my time for something that didn't pay off. Oh well, that's what you get for not researching thoroughly.

MentholMoose

zerglings wrote: »

I found out that my IDE DVD/RW was not compatible with ESXi.

It's probably the PATA controller that's unsupported, not the drive itself. ESXi works fine with PATA CD/DVD drives, on supported controllers, in my experience. Intel and AMD dropped PATA support a long time ago in their chipsets, so the PATA port on most recent motherboards is handled by some add-on controller, often from JMicron or VIA (I check the specs on the M4A88TD-V EVO and it uses a VIA controller for the PATA port).

zerglings wrote: »

I found out while I was transferring all my files to VMFS formatted drives that the maximum throughput of ESXi is not that high, hovering around ~40 MB/s and I was transferring about 1TB of data.

I guess you are transferring it over your network. There may be some bottleneck somewhere, plus using unsupported hardware may not give the best performance.

zerglings wrote: »

I found out that since ESXi formatted my RAID 1 1.5TB drive and RAID 1 640GB to the default block size of 1MB automatically, I was limited to only 256GB of data. I was scratching my head why I was getting this I/O occurred error. After reading several forums, people were suggesting to install FastSCP or WinSCP to transfer files. Did that, but finally the I/O occurred error that I was getting was because of no space on the drive
- I found out that when ESXi formatted my 1.5TB to 1MB block size, I couldn't take it out from storage section. Tried different things suggested on VMware forum and nothing worked except the guy who suggested to format it to other file system and ESXi should see it as a new drive.

This is where those Configuration Maximums come in handy.

zerglings

MentholMoose wrote: »

It's probably the PATA controller that's unsupported, not the drive itself. ESXi works fine with PATA CD/DVD drives, on supported controllers, in my experience. Intel and AMD dropped PATA support a long time ago in their chipsets, so the PATA port on most recent motherboards is handled by some add-on controller, often from JMicron or VIA (I check the specs on the M4A88TD-V EVO and it uses a VIA controller for the PATA port).

Good to know. Didn't know that they stopped supporting it.

MentholMoose wrote: »

I guess you are transferring it over your network. There may be some bottleneck somewhere, plus using unsupported hardware may not give the best performance.

Have you tried it on supported hardware? If so, what was your file transfer rate? I am thinking the bottleneck was my old P4 based server.

MentholMoose wrote: »

This is where those Configuration Maximums come in handy.

I knew from the start about the block sizes. I was researching about it before I actually built the server. However, the ESXi on flash drive that I created automatically initialized all the drives connected to my Perc 5i and used the default block size of 1MB. I never cared to look until I experienced the issue.

I quite happy with my setup so I guess ESXi will be on hold until I have use for it. Unfortunately, I do not have any exposure with VMware at work enough to be concentrating on learning that. I'd use my spare time on IE studies after this "weekend project" is done.

MSNinja

zerglings wrote: »

- I found out that since ESXi formatted my RAID 1 1.5TB drive and RAID 1 640GB to the default block size of 1MB automatically, I was limited to only 256GB of data. I was scratching my head why I was getting this I/O occurred error. After reading several forums, people were suggesting to install FastSCP or WinSCP to transfer files. Did that, but finally the I/O occurred error that I was getting was because of no space on the drive

Block size does not determine how much data each vmfs LUN can hold. It only represent how large the .vmdk files (aka virtual hard drives or files) can be on the VFMS volume.
So on a 640GB volume you should be able to keep almost 2.5x 250 GB virtual disks.

Here is a great kb from VMware VMware KB: Reformatting the local VMFS partition's block size in ESX 4.x (post-installation) regarding setting the block size during installation.

zerglings

MSNinja wrote: »

Block size does not determine how much data each vmfs LUN can hold. It only represent how large the .vmdk files (aka virtual hard drives or files) can be on the VFMS volume.
So on a 640GB volume you should be able to keep almost 2.5x 250 GB virtual disks.

Are you saying that WinSCP was lying to me when it said something in the lines of "The disk is full"?

MSNinja wrote: »

Here is a great kb from VMware VMware KB: Reformatting the local VMFS partition's block size in ESX 4.x (post-installation) regarding setting the block size during installation.

I did that as well but for whatever reason, it just doesn't delete the 1.5TB drive. I had to format the HDD on a Windows before I could redo the block size to 8MB. When I did the 8MB, I was able transfer over 256GB worth of data.

rscarson

I understand necromancy is frowned upon, but I thought I might save some trouble for any owner of an m4a88td who like me, has been banging his head against a wall for hours.

ESXi 5 and above require bios version 2301 or above. The suggestion to downgrade the bios is only for v4.

Sorry again for digging up this thread, but I couldn't find this info anywhere else, and this thread is first in the google search results for m4a88td-v esxi 5.5