Backup Question - insufficient speed

bobloblaw

I'm backing up around ~10tb of data. It's just too slow over gigabit ethernet. I was wondering if I just took a cat6 cable and plugged it straight in from the backup server to our file server, would that speed up the transfer rate?

I've used an old antiquated tape backup for years as a backup solution because I haven't needed anything else. Now we've very much outgrown it, and the thing just can't keep up anymore. Looking for a different backup solution, and I'm pretty ignorant as to what is currently out there. Leaning towards a big cheap scalable HP NAS. It's amazing how cheap a 90TB system is these days.

networker050184

To know if that will make any difference you'd need to know what is causing it to be so slow. Or if it's even "slow" or that's just the rate you're going to get.

scaredoftests

Get a new tape backup! Or get a NAS. That is what I have at home (I am assuming this is for home use). I have a 2TB NAS drive and I backup to there.

bobloblaw

It took me ~23 hours to backup ~1.3tb of around 700k files. It's on the same switch too. Backing up straight to disk in this scenario.

elTorito

In 23 hours, you should be able to backup much more, in theory. Theory being that you're backing up mainly large files that can be written sequentially. When working with many small files, you're going to see your throughput plummet. There's no way around that, even if your backup network is 10g or 40g.

From your description (700k files @ 1.3 TB), it doesn't seem like there's an inordinate amount of small files, though. So perhaps there's something else that's bottlenecking your performance.

When you're backing up to disk, what software are you using? What's the disk composition on the backup server or the storage attached to that backup server? Are there enough local computing resources (CPU, memory) to support that I/O passing through?

What's the performance if you use a multithreaded Robocopy/Xcopy to copy the same amount of files from the file server to a Windows Server, using the same network equipment inbetween? If you can squeeze considerably more out of this, it rules out the switch being the culprit.

I'd strongly suggest finding the current problem first, before investing in a whole new backup system, lest you find your shiny new equipment being held back by the bottleneck that you didn't find. Explaining to management that the € 100.000 investment didn't bring the goods can be a bit ... awkward

bobloblaw

Thanks for the advice. I'll give robocopy a try. I'm certainly not redlining the NIC.

bobloblaw

Robocopy cut it down to 8 hours compared to 23. Clearly an issue of the backup software I was using to test. I'm going to try some different software now.

bobloblaw

Speed is still meh for the current 10tb I need to backup. I'm going to try some backup agents, but I'm very tempted right now just to put two 10GbE cards in both machines and direct connect.

iBrokeIT

Are you using a backup solution that makes use of deduplication? Might be something to look into if only 5-10% of your data is changing and you don't need to backup other ~90% on a regular basis.

bigdogz

If this is a personal / home environment, you may just shoot for the NAS.

If this is for a business, you may want to ask your boss what it's worth for backup systems.

You may not have been bit by the "My backups don't work bug". Tread lightly.

You may want to look at this from a DR perspective.

Tape is old and not as reliable as disk to disk. After some time the physical media of the tape goes bad or is inconsistent during a restore.

Make frequent snapshots / images of your servers.
Make frequent restores from random points in time. This is where you will see if your old tape system is reliable... YMMV.

You may also want to look at the NIC. Microsoft has a wonderful feature called Power Management. ...SMH....It is a default setting to be on all Windows PC's and servers. If it is on for the NIC, this may slow your speed of your NIC.

If you run a Disk to Disk backup this will help you manage your environment a great deal faster. If you want specific names I can look some up if you wish.

bigdogz

.....eVault is one of the systems that I have used and found it to be very reliable.

Good Luck!!!

bobloblaw

Appreciate it.

Trying Backup Ex 15 now for a full. It's going at a much better clip than the client-less/agent-less backups. I'm pretty sure it has dedup capabilities as well. Just trying to do due diligence before pulling the trigger on a big NAS.

I'm still leaning towards getting a couple 10GbE cards for a direct connect. I have ~15 on site servers, but I'm really only concerned with the big boy that currently has ~10tb on it. Even if it's a wash, I definitely can blow $600 for a couple cards to test a direct connect before I justify a $20k+ scalable ~100tb NAS.

EMC quoted us ~$170k for what we need. I told my boss "No way I can't beat that handily." It's proved... interesting