This has turned into a pricey hobby...

Deathmage

I just spent $500 extra on the home cluster in one fell swoop.

$500+ dollars for memory for two of my hosts:

Refurbished: Micron 48GB (6x 8GB) PC3L-10600R ECC Registered DDR3 1333MHz Server Memory Kit for Dell PowerEdge R610 R620 R710 R720 R810 R820 R910, (6x Model MT36KSZF1G72PZ-1G4) - Newegg.com

But now the cluster will have 168 GB's of RAM across 4 hosts w/ the 8 x E5520 processors.

Next month is a good number of Velocoraptor 10k drives and 4 SSD's to make a vSAN across the hosts and either a FCoE or FC network just for vSAN storage.

I may just make a Tier 1, 2 and 3 Storage fabric for the home-cluster between vSAN and the two other NAS's since one is a RAID 5 and the other is a RAID 10.

Konflikt

nice setup. what's the power consumption? what about the neighbors and the noise?

Deathmage

Konflikt wrote: »

nice setup. what's the power consumption? what about the neighbors and the noise?

What electric, I got solar and geothermal heating/cooling.

fmitawaps

How do you "refurbish" a memory stick? Do they desolder all the chips and test / replace them, and solder it all back together?

Or do they send the RAM sticks to China and some kid cleans them, runs them through a memory stick tester, and if they pass (and probably the ones that don't pass too, for more profit), put a "refurbished" sticker on it and send it back to the USA?

kj0

fmitawaps wrote: »

How do you "refurbish" a memory stick?

Run an Eraser over the terminals and shine them up


Trev. Maybe do up a blog on your full setup? Something like Chris Wahl's? Home Lab - Wahl Network

Essendon

I plonked $3000 approx into a new home lab last week. Let's see if it can rival yours Trev I'll do a blog post about it when I have some spare time.

No_Nerd

What did you get for 3k ?

Deathmage

No_Nerd wrote: »

What did you get for 3k ?

A midget reactor to power it.. like bit size, Homer Simpsons shoe box reactor, made with Duff.

Deathmage

@kj0 @Essendon ....yes a home-lab write up on the blog might be in order. Doing the vSAN next month.

No_Nerd

Sounds good . for 3K I will usually buy two pairs of shoes

jibbajabba

I got a R710 with 144GB and dual E5650, Synology Slim with 4x480GB SSD and an SG300 ... Fail to see the need for two hosts ... Am I getting old ?

joelsfood

I don't think so, jibbajabba. I just run one host and then nested on my desktop

Iristheangel

Yep. Labbing is expensive but on a happy note, TAX WRITE OFF :P

I've actually had to move some of my servers into a colo. I found one at One Wilshire in LA that gave me a /29, 2 AMPS, 2U of space, and a 100Mb pipe for $69/month. Was way cheaper than trying to power and cool the only room in my house that I could stick it in without hearing it from the bedroom which also happens to be the hottest room in the house. The other ones I keep at work but it's a pain sometimes because it's through a proxy and accessing it from the internet or giving other people access isn't really possible. Depending on your situation, it might be cheaper to look into housing it somewhere else if it ends up getting too expensive to house it all at home.

Though my situation is a bit different since a lot of my labs are 100% virtual so one or two beefy servers does it for me.

Deathmage

joelsfood wrote: »

I don't think so, jibbajabba. I just run one host and then nested on my desktop

jibbajabba wrote: »

I got a R710 with 144GB and dual E5650, Synology Slim with 4x480GB SSD and an SG300 ... Fail to see the need for two hosts ... Am I getting old ?

I wanted to make a vSAN cluster and it said I needed 3, so I got 4. One is just a Development server. But I also have 6 Windows 2008 R2 domains, and I use the Cisco lab to make 6 different vlans per domain (one 2960G per two hosts with Quad 1G EC per host (with 6 vlans being tagged to the trunk) with eight 1G's EC to Distro layer per 2960G), But I have 12 vlans total for VDI's, server, desktops, backups, wireless, and one I'm doing right now for a DMZ. I got a ton of VM's, and I'm going for a Large Enterprise landscape in the home-lab.

One thing I have to state, is now that I've been making 6 separate TCP/IP stacks in esxcli on the the hosts for each VMKernal, the performance of LRO seems to be much higher for my iSCSI VMKernel, as does vMotion and svMotion on their respective stacks. TSO now called LPO also seems to work better, but for good measure I have Jumbo also enabled even though I think TSO/LRO do just fine. I think Custom TCP/IP Stacks for 6.0 is a nice addition.

I want to start dabbling in vCloud Director, thanks jibba! - so I'm using the FR network to mimic a WAN connection and it works at the network layer, so in theory once I start putting VDI's on the Raspberry Pi 2's in Ubuntu I 'should' haven't figured it out yet, I should be able to use the FR fabric to connect to the 3750G Core/Distro/Access layer which has the ESXi cluster that has vCloud Director. Goal is to make a few Windows Domains over the WAN.

It sounds complicated but I'm engineering it all piece by piece. I think it should be done by a few engineers whom are dedicated to systems/networking/storage/VMware for each, so meh I'm doing it all, so it take time. Once I'm all done I'll make a blog posting, it might be a few dozen pages long. - But I'm using the complete lab for the Cisco, Microsoft, VMware studies I'm doing as I go along.



I got two NAS's one is just a 4 bay filer and the last one is a 6 bay filer in RAID 10 with two 120 GB SSD's.

But in all honesty, I'll probably just have two on, but having a 3rd has it's purposes...

Iristheangel wrote: »

Yep. Labbing is expensive but on a happy note, TAX WRITE OFF :P

I've actually had to move some of my servers into a colo. I found one at One Wilshire in LA that gave me a /29, 2 AMPS, 2U of space, and a 100Mb pipe for $69/month. Was way cheaper than trying to power and cool the only room in my house that I could stick it in without hearing it from the bedroom which also happens to be the hottest room in the house. The other ones I keep at work but it's a pain sometimes because it's through a proxy and accessing it from the internet or giving other people access isn't really possible. Depending on your situation, it might be cheaper to look into housing it somewhere else if it ends up getting too expensive to house it all at home.

Though my situation is a bit different since a lot of my labs are 100% virtual so one or two beefy servers does it for me.

Last years write off was over 2000, it was nice.

I keep the servers and lab in the basement, it's constantly 55 degrees and when it's on it heats the basement...


my problem is I spend so much time labbing on the home-cluster making it how I want it, I don't study as much. My experience though far outpaces my certifications so my home-lab is my trophy of my knowledge.

kj0

One physical host (AMD whitebox I built). I paid $600 for CPU, motherboard, RAM and PSU. I'm using a spare computer as NAS with Openfiler. Once I get my RAID Controller working again (Having issues reapplying firmware) then I should have both my SSDs working in my desktop and I can run up another site.

Networking side I use some Netgear unmanaged Gigabit switch that supports VLANs and using a Junipoer SSG 140 to supply my VLANs. (Both which I picked up from work)

Can't really expand much more though, need to keep the amount of room minimal.

Deathmage

kj0 wrote: »

One physical host (AMD whitebox I built). I paid $600 for CPU, motherboard, RAM and PSU. I'm using a spare computer as NAS with Openfiler. Once I get my RAID Controller working again (Having issues reapplying firmware) then I should have both my SSDs working in my desktop and I can run up another site.

Networking side I use some Netgear unmanaged Gigabit switch that supports VLANs and using a Junipoer SSG 140 to supply my VLANs. (Both which I picked up from work)

Can't really expand much more though, need to keep the amount of room minimal.

Well if you want, once I get vCloud Director up and working if you ever want to use the cluster for testing let me know. The cluster is always on during the weekend from 6pm Friday until about 10 pm on Sunday EST. I could setup a SSL VPN on the Sonicwall to only one management vlan that is used for a segmented ESXi cluster in vCloud.

JockVSJock

Iristheangel wrote: »

Yep. Labbing is expensive but on a happy note, TAX WRITE OFF :P

How are you writing this off of your income taxes?

joelsfood

Educational/business expenses unreimbursed by your employer. Even easier if you're self employed. Not that 2k in hardware knocks 2k off your tax debt, of course. How much you get out of it depends on your tax bracket.

JockVSJock

joelsfood wrote: »

Educational/business expenses unreimbursed by your employer. Even easier if you're self employed. Not that 2k in hardware knocks 2k off your tax debt, of course. How much you get out of it depends on your tax bracket.

I could see the self-employment for the writeup, but if you working for someone...

I've never taken up my employer to pay for my cert, I've always paid for it out of my own pocket. I don't want to be stuck or obligated to stay there for two years when I can't get promoted.

So the Red Hat Cert Voucher that I paid for last year, I can put that cost on education expense? I don't own or operate my own business.

The Cisco router/switches that I bought about 10 years ago can also be seen as an education writeoff too?

joelsfood

You'd have to consult your local tax expert to say for sure (or look through the directions on turbotax). Here are a few more details from the IRS though

https://www.irs.gov/taxtopics/tc513.html

Iristheangel

JockVSJock wrote: »

How are you writing this off of your income taxes?

I always make sure to do a little bit of consulting through the year to help justify my $15-20K of tax writeoffs through the year and show that it isn't just a high-priced hobby. Once you get past a certain amount of writeoffs, you need to justify it and show that it's not just a bunch of self-improvement but that you're turning that into revenue. If you find yourself writing off a lot, you can always set up a DBA and do a couple other things to show that you are serious to the IRS. If you have some consulting gigs that will give you a 1099, even better. If you find yourself having less write-offs, even companies like Onforce will give you a 1099 if you're doing contracting work through them and you might not need to start a DBA or get a tax identification number for your consulting business. Like Joel said, consult your tax adviser to find out what works best for you or if you're really going to benefit from it.

alias454

My work dropped 10k on parts for my VSAN POC project -- Running it on 3 HP DL380 G6's for now.

Deathmage

I write off all my IT study material for the year: books, tests, hardware. Everything I pay for and is taxed at some point can be a write off, as per my tax person. Since doing it I have yet to pay for taxes.

Deathmage

Just nabbed one of these off ebay.

http://www.netgear.com/business/products/switches/unmanaged-plus/10g-plus-switch.aspx#tab-overview

This will suffice the needs of a 10GbE SFP+ for the QNAP NAS's 10GbE SFP. then I can run short copper Cat 6/7 from the ESXi's to the switch. This will comprise of the vSAN and iSCSI Storage Fabric for the 6 bay filer.

The 4 bay QNAP NAS will still be on a 1G 2960G switch with Quad EC per host.

Wonder if iSCSI and vSAN can sit on the same subnet, in a home-lab.

kj0

Deathmage wrote: »

Well if you want, once I get vCloud Director up and working if you ever want to use the cluster for testing let me know.

Thanks Trev. Should be all good - I'll just move into using our spare gear at work. It's only when we run out of spare rooms (I think my drums will lose their room first before my lab)

Deathmage

kj0 wrote: »

Thanks Trev. Should be all good - I'll just move into using our spare gear at work. It's only when we run out of spare rooms (I think my drums will lose their room first before my lab)

Priorities, drums are good for play, but how does that compare to VMware.

jibbajabba

Deathmage wrote: »

Priorities, drums are good for play, but how does that compare to VMware.

Personally I sometimes could smack the crap out of vmware

Deathmage

jibbajabba wrote: »

Personally I sometimes could smack the crap out of vmware

This is why we have home-lab so when you do it no one can press charges or get offended.

Deathmage

here is a picture from the QNAP NAS (upgraded the memory to 16 GB of DDR3L too) at home, it has a RAID 10 made up of (4) 4 TB WD RED drives with a Samsung Evo 120 GB SSD for Cache Acceleration with a Bonded 10GbE connection to the Netgear switch that arrived last night into the Dell R610's on the Intel Dual 10GbE x520-T2 Converged Network Card into a Windows 8.1 VM.

I only bought one card so far cause I wanted to see if it would work and boy does it work....look at these speeds with Jumbo Frames enabled....Going to use this channel for vSAN and for iSCSI. I'll have plenty of bandwidth.

The below is a file transfer from the NAS to the VM.

I've found no difference if I use Cat5e or Cat6, so I'm just keeping it at Cat6 for now.