iSCSI in a big nutshell

jthunderbird

Hello all,

So I usually try not to waste people's time and can typically find everything I need with a search... but the knowledge I am seeking now is freaking eluding me.

I am a network guy, Cisco all the way... but am trying to expand my knowledge to virtualization (want to focus on VMWare and Hyper-V). All this being said, I have worked VERY little with any real storage solution outside of configuring my end of an etherchannel going to a virtual switch.

Myself and another CCNP buddy of mine are signed up for the Stanly summer course, but neither of us are in it just for the cert... we want to know what we are doing and become good at it. So I have a 7' server rack (mostly used for training friends/family on more basic things) with some switches, routers and servers. Concerning VM, it has 2 2950 III servers with 16GB of RAM each and 4 1TB drives each. I understand the concept of using iSCSI to basically have a locally connected drive that is shared within my cluster.

However, I am caught up on figuring out how to implement this. My plan is to grab a Dell MD1000 off ebay (I dont mind spending some cash on my lab obviously), throw our 8TBs of SATA drives in there, a couple 15k 73GB drives in each 2950 future ESXi/Hyper-V host, and use this as my iSCSI solution.

Tell me how please

Does the MD1000 need another server to manage it, then my 2950s connect via IP that way? Or does the MD1000 have some proprietary software and is standalone? The array I will likely buy has 2 EMM cards. Do I need to download a software and run it on the storage server and storage clients (intiiators)?

I promise I have done research, but most are explaining the basics of what iSCSI is or explaining the software implementation... I need the knowledge in the middle.

googol

MD3000 is a DAS, a direct attached storage. You need network cards and present shared storage to both the servers for a true virtualization setup. You would need a server like windows storage to present the DAS storage out on its own NICs if you wanted to go with MD3000. Look for a model that supports specifically iSCSI, then they will have built in software/firmware on them that presents it via iSCSI and has NICs on a controller...

kriscamaro68

You could just build a computer with a bunch of ssd's and set them up in a storage pool in server 2012 and then create an iscsi volume and present it to the hosts. You will want multiple nics setup on the 2950's so that you can setup MPIO on the windows side.

emerald_octane

I would get a prosumer grade nas from Netgear, QNAP etc with iSCSI and load these up with your storage, then get tiny but decent SSDs for your host boxes. SD cards would work but you'd need to find an internal interface.

Build up your networking stack (management, vm traffic, storage traffic). Storage and vm traffic is L2, management traffic is/can be L3.

Install SD/SSD cards in your ESXi hosts and build them out into the environment, connect your management NICs using the host interface and use the vSphere Client to finish configuration (vm and storage traffic).

-optional- deploy vCenter for cluster type management

Connect iSCSI storage to storage subnet, create lun, configure iSCSI target.

You have several choices to configure the storage:
1.Connect to Host using Software iSCSI initiator
2. Connect to Host using HBA (i.e. Fibre Channel Deployments)
3. Connect to VM using Software iSCSI Initiator

1 and 3 can be done natively , 2 requires a more expensive card, but the value add is faster processing.

jthunderbird

googol wrote: »

MD3000 is a DAS, a direct attached storage. You need network cards and present shared storage to both the servers for a true virtualization setup. You would need a server like windows storage to present the DAS storage out on its own NICs if you wanted to go with MD3000. Look for a model that supports specifically iSCSI, then they will have built in software/firmware on them that presents it via iSCSI and has NICs on a controller...

Seems the models that specifically support iSCSI out of the box are very pricey. Converting a DAS MD1000 seems like it wouldnt be that tough... so I am assuming I will need to attach the MD1000 to a management server via a SAS cable and run one of the software iSCSI solutions? Does VMWare have a built in "initiator" or do I need to run software on that server as well?

jthunderbird

emerald_octane wrote: »

I would get a prosumer grade nas from Netgear, QNAP etc with iSCSI and load these up with your storage, then get tiny but decent SSDs for your host boxes. SD cards would work but you'd need to find an internal interface.

Build up your networking stack (management, vm traffic, storage traffic). Storage and vm traffic is L2, management traffic is/can be L3.

Install SD/SSD cards in your ESXi hosts and build them out into the environment, connect your management NICs using the host interface and use the vSphere Client to finish configuration (vm and storage traffic).

-optional- deploy vCenter for cluster type management

Connect iSCSI storage to storage subnet, create lun, configure iSCSI target.

You have several choices to configure the storage:
1.Connect to Host using Software iSCSI initiator
2. Connect to Host using HBA (i.e. Fibre Channel Deployments)
3. Connect to VM using Software iSCSI Initiator

1 and 3 can be done natively , 2 requires a more expensive card, but the value add is faster processing.

I understood about 2/3rds of that I think... Just got off the phone with a VCP friend of mine who is pretty well versed in this and he said he will come by next weekend and help with the solution but gave his thoughts at this moment... basically saying I should just use a cheap server (I have a 2850 laying around the lab area somewhere), install my drives and run a software (think I am going with starwind)...

This seems a little too easy and I am certain I am missing steps, but I am going to give it my best shot... knowing nothing about storage to trying to get a decent admin level understanding of storage in 2 days is taxing!

thenjduke

It really is not all that bad. Starwind is nice to setup. Just get old server put your drives in it and install any Windows OS. Do make sure to have at least three network cards two for the ISCSI network and the one to RDP to the server. I usually seperate my lab in different vlan on a Cisco switch and make sure have Gig Ethernet or better switch and cards.
Then follow this.
Install and configure StarWind Iscsi San for Vmware Esxi | Do The Job

kj0

I did a blog up last month on how to set up a nested OpenFiler iSCSI. You can use FreeNAS or Starwinds in a similar fashion. The idea is that you can get any box, or even a VM and run the software. I'm running it inside a VM, but you just need to configure it at the openfiler webclient end so that you LUN is presentable and then you can go ahead and set up the iSCSI connection within vSphere.

ReadySetVirtual: Storage Pt 2 - Set up Nested iSCSI Target/SAN using OpenFiler

I would suggest that if you want to learn, get your hands on Mastering vSphere 5.5 by Scott Lowe and Nick Marshall.

If you're looking at an MD3000, it is the MD3000i that is iSCSI, We are just replacing ours at the moment. (Holding for a dev environment though. )

Essendon

Or just have Starwind iSCSI SAN software running on a Windows 2008 R2 VM - works just like the real thing. No need to burn power or money for hardware, just one beefy machine will do.

jthunderbird

Ok thanks for the advice fellas. I like to simulate real world as much as possible and do not mind the extra $$ to do so(within reasonable means)... thats why I am trying to stay away from a nested vm environment, even if it essentially the same thing.

Building ESXi in Workstation is as simple as pointing to an IOS and clicking next a few times whereas building it on a physical box can bring a couple complications... and I like complications... they force you to learn.

When I made this post, I had been studying storage for about a day and a half... now its been about 2 and a half days and I have a better overall view. Cant wait to actually get into it and build my iSCSI SAN tonight and test it out. ESXI servers will not be coming til tomorrow, but the home lab should be ready and waiting... and to those suggesting the "Mastering VMWare" book... like I said, I usually ***** these forums pretty thoroughly and have read the praises of this book many times... it currently already resides in my lab

Thanks everyone for your input. Always open to new views.

**EDIT: why is the word t roll blanked out? Maybe I should have said "peruse" instead

jibbajabba

Essendon wrote: »

Or just have Starwind iSCSI SAN software running on a Windows 2008 R2 VM -

Or skip Starwind and simply install the native ISCSI Target on 2008R2

Download Microsoft iSCSI Software Target 3.3 from Official Microsoft Download Centre

Or for proper performance even 2012R2

jthunderbird

jibbajabba wrote: »

Or skip Starwind and simply install the native ISCSI Target on 2008R2

Download Microsoft iSCSI Software Target 3.3 from Official Microsoft Download Centre

Or for proper performance even 2012R2

Would certainly be nice to not have to resort to a third party application... how easy is this to install and configure? Starwinds looks super easy with a nice GUI... I am a Cisco guy and love command line, but when it comes to windows, I need my GUIs

tstrip007

Microsoft iSCSI is very simple and easy. Heres a how to...

Building the Ultimate vSphere Lab – Part 10: Storage | Boerlowie's Blog

NISMO1968

StarWind is a definite winner for production scenario just because it is: a) fault tolerant active-active clustered out-of-box (MS target or newer SoFS need some back end storage or single target and single SMB 3.0 folder would become a single point of failure) b) faster as StarWind is RAM & flash cached and does ODX (?) and is active-active (MS target is non-cached and can do active-passive only) c) StarWind can run on the same hardware where hypervisor runs similar to VMware VSAN scenario (MS target, SMB and SoFS need dedicated hardware and don't work in a loopback by-design)But for test & development a), b) and c) are not critical so you may use what's easier and nicer for you to use I do use FreeBSD-based ZFS appliances.P.S. c) you can run MS target in a loopback for sure with client VM on the same host it's just can be hardly called a production scenario )

jthunderbird wrote: »

Would certainly be nice to not have to resort to a third party application... how easy is this to install and configure? Starwinds looks super easy with a nice GUI... I am a Cisco guy and love command line, but when it comes to windows, I need my GUIs

jonny72

As you're planning to learn Hyper-V at some point, install Server 2012 R2 on one of the 2950's, load it with hard drives and use that to provide iSCSI targets. No point spending money and time on other technology that you're not interested in learning about.

If you're itching to spend some money, get some 1950's instead to use as ESXi hosts - no need for storage in them, just run ESXi off a USB / SD card.

NISMO1968

Microsoft iSCSI is dead. MSFT is actively shifting in SMB 3.0 direction with all corresponding technologies (Clustered Storage Spaces as a back end, Scale-Out File Servers, SMB Direct, SMB Transparent Failover etc).

See Jose Bareto great blog about future of MS built-in storage (that does not affect third-party software though):

Windows Server 2012, File Servers and SMB 3.0

You can deploy VMs with a free Hyper-V from DAS just fine. Also you can run Hyper-V from USB. No point to switch to a free ESXi being shorter in features list (no VM HA, no VM migration etc). See (good discussion about Hyper-V Vs. ESXi and booting and running Hyper-V from USB, accordingly):

Hyper V vs ESXi - Spiceworks

Run Hyper-V Server from a USB Flash Drive

Good luck!

jonny72 wrote: »

As you're planning to learn Hyper-V at some point, install Server 2012 R2 on one of the 2950's, load it with hard drives and use that to provide iSCSI targets. No point spending money and time on other technology that you're not interested in learning about.

If you're itching to spend some money, get some 1950's instead to use as ESXi hosts - no need for storage in them, just run ESXi off a USB / SD card.

kj0

NISMO1968 wrote: »

. No point to switch to a free ESXi being shorter in features list (no VM HA, no VM migration etc). See (good discussion about Hyper-V Vs. ESXi and booting and running Hyper-V from USB, accordingly):

What are you basing this on? You get 60 days free full Enterprise Plus. Plenty of time for Labbing, and you are more likely to reinstall over that study time, so you can start again. MY 5.5 Eval has made it into it's 4th month after a reinstall.

jthunderbird

I planned on getting an eval of Server 2012 R2 for Hyper-V so probably not a bad idea to go ahead and download it I guess.

Anyway, I went ahead and just stuffed the big hard drives in the 2950 ii, loaded Server 2008 and installed Starwind. Was really easy to make a huge img file as my target and VMWare recognized it no problem.

Only issue I am having now is speed. It takes forever to move data to and from this SAN. I blame it on the 3550 in the middle... 100mb ports and no jumbo frame support. Since this is an unrouted vlan, I went ahead and ordered a d-link unmanaged 5 port switch that is gb links and supports 9k jumbo frames... so hopefully my speed will pick up nicely during this weekend's labbing.

Thanks for the input guys... my main concern initially was just getting a grasp on iSCSI... I think Ive got a firm one now!

NISMO1968

Talking about people who don't wipe off their hypervisor and roll new one on a routine basis )

kj0 wrote: »

What are you basing this on? You get 60 days free full Enterprise Plus. Plenty of time for Labbing, and you are more likely to reinstall over that study time, so you can start again. MY 5.5 Eval has made it into it's 4th month after a reinstall.

NISMO1968

Oh, you really need at least 1 GbE for iSCSI (and pretty much everything else these days...).

jthunderbird wrote: »

I planned on getting an eval of Server 2012 R2 for Hyper-V so probably not a bad idea to go ahead and download it I guess.Anyway, I went ahead and just stuffed the big hard drives in the 2950 ii, loaded Server 2008 and installed Starwind. Was really easy to make a huge img file as my target and VMWare recognized it no problem.Only issue I am having now is speed. It takes forever to move data to and from this SAN. I blame it on the 3550 in the middle... 100mb ports and no jumbo frame support. Since this is an unrouted vlan, I went ahead and ordered a d-link unmanaged 5 port switch that is gb links and supports 9k jumbo frames... so hopefully my speed will pick up nicely during this weekend's labbing.Thanks for the input guys... my main concern initially was just getting a grasp on iSCSI... I think Ive got a firm one now!

jthunderbird

NISMO1968 wrote: »

Oh, you really need at least 1 GbE for iSCSI (and pretty much everything else these days...).

Yea and I knew that before getting started, but needed to go through the labbing with what I had to at least get the theory in my head... now that I understand iSCSI, I can (and have) explain it and get others to understand... its not that hard, but I needed this little lab to just work it out on my own.

My GB switch should be here today so my iSCSI network should speed up quite a bit... which is needed, because it is beyond slow right now.