Options
Large storage needs
We will be setting up direct storage, consisting of 450GB drives, in RAID 5 to store and serve 4TB of aerial photo projects. We need almost all the disk space so I was wondering just how full is too full? At what percentage full does performance go down. Storage is not my specialty as you can probably guess.
I'd like to configure the 12 disk enclosure as one RAID 5 with 11 disks and a spare. Others feel that 11 drives is too many and that we should configure it as 2 5 disk RAID 5 arrays. Part of their logic is that if we lose 2 disks at the same time we only lose half the data. And therefore only need to restore that half. Can anyone point me to a white paper on best practices for setting up storage?
The datacenter we are forced to use has never dealt with a project like ours. Their business model is not designed for large volumes of data. The yearly cost for storing our data on the data center SAN is such that the direct storage pays and support pay for for itself in less than a year. Don't you just love government
We also can't aford the $3.50 per GB to backup the data to tape so we are purchasing a second inclosure which will be filled with 1TB SATA drives. We will be using 2nd enclosure for backing up the data. This is not how we want to do things but it is our only option short of not doing the project in the first place. BTW, the nature of the data is that it never changes.
I'd like to configure the 12 disk enclosure as one RAID 5 with 11 disks and a spare. Others feel that 11 drives is too many and that we should configure it as 2 5 disk RAID 5 arrays. Part of their logic is that if we lose 2 disks at the same time we only lose half the data. And therefore only need to restore that half. Can anyone point me to a white paper on best practices for setting up storage?
The datacenter we are forced to use has never dealt with a project like ours. Their business model is not designed for large volumes of data. The yearly cost for storing our data on the data center SAN is such that the direct storage pays and support pay for for itself in less than a year. Don't you just love government
We also can't aford the $3.50 per GB to backup the data to tape so we are purchasing a second inclosure which will be filled with 1TB SATA drives. We will be using 2nd enclosure for backing up the data. This is not how we want to do things but it is our only option short of not doing the project in the first place. BTW, the nature of the data is that it never changes.
Comments
-
Options
Tyrant1919
Member Posts: 519 ■■■□□□□□□□
How many users are accessing this data at once?A+/N+/S+/L+/Svr+
MCSA:03/08/12/16 MCSE:03s/EA08/Core Infra
CCNA -
Options
blargoe
Member Posts: 4,174 ■■■■■■■■■□
If losing 2 disks is a concern, you could just set up RAID6. Writing performance would go down, but if you're primarily serving static content and they're not changing it, shouldn't be a big deal.
Depending on how many people are using this enclosure, even using SATA on the first enclosure might be OK.IT guy since 12/00
Recent: 11/2019 - RHCSA (RHEL 7); 2/2019 - Updated VCP to 6.5 (just a few days before VMware discontinued the re-cert policy...)
Working on: RHCE/Ansible
Future: Probably continued Red Hat Immersion, Possibly VCAP Design, or maybe a completely different path. Depends on job demands... -
OptionsTyrant1919
Member Posts: 519 ■■■□□□□□□□
That's what I was getting at. If the number of users is very low, I'd go to 1TB or 2TB drives. Hell, 4 2TB drives in raid 5 and you're set. 5 Drives in raid 6 if you want two drive protection. But if you're backing it up, depending on your needs, the extra protection may not be warranted.A+/N+/S+/L+/Svr+
MCSA:03/08/12/16 MCSE:03s/EA08/Core Infra
CCNA -
Options
tiersten
Member Posts: 4,505
Doesn't affect it. You've got static data anyway.We need almost all the disk space so I was wondering just how full is too full? At what percentage full does performance go down.
How certain are you that the data will never change? Does this include whatever else is running off the data? I hope that you're still backing it up on to something other than another enclosure in the same datacenter though.We also can't aford the $3.50 per GB to backup the data to tape so we are purchasing a second inclosure which will be filled with 1TB SATA drives. We will be using 2nd enclosure for backing up the data. This is not how we want to do things but it is our only option short of not doing the project in the first place. BTW, the nature of the data is that it never changes.
What level of performance do you need from this? RAID5/6 requires a large number of spindles for decent performance. If its only a handful of people accessing these files at any given time then a low number of drives will be fine but if there are more then you'll need to consider performance.
If you're concerned enough to warrant using RAID, check that the drives you're buying are actually rated to run 24x7. Regular desktop drives whilst cheap aren't actually rated for that. -
Options
dynamik
Banned Posts: 12,312 ■■■■■■■■■□
Have you thought about getting larger drives and going with RAID-10? -
Options
astorrs
Member Posts: 3,139 ■■■■■■□□□□
Whenever you're calculating a storage solution you need 3 things.
1st is the space required and approximate growth - it sounds like you have an handle on that.
The second is IOPS (I/Os per second), how much performance do you require from the device - hard numbers are best here - where is the data currently stored and can you provide some number from that? (if you need help gathering this we can give you some pointers if you tell us where its currently stored)
Finally you need to understand the availability requirements, again you have an idea, but putting it in business terms is better - e.g. "Recovering from a backup in 24hrs time and losing a couple days worth of data is okay" vs "Downtime on this data is unacceptable and will cost the company X dollars per second in lost business"
Without those 3 things you are just guessing that a particular solution will meet your needs. -
Optionsdynamik
Banned Posts: 12,312 ■■■■■■■■■□
Have you thought about getting larger drives and going with RAID-10?
-
Optionsblargoe
Member Posts: 4,174 ■■■■■■■■■□
How about RAID 10, has anyone suggested that yet?IT guy since 12/00
Recent: 11/2019 - RHCSA (RHEL 7); 2/2019 - Updated VCP to 6.5 (just a few days before VMware discontinued the re-cert policy...)
Working on: RHCE/Ansible
Future: Probably continued Red Hat Immersion, Possibly VCAP Design, or maybe a completely different path. Depends on job demands... -
Optionsdynamik
Banned Posts: 12,312 ■■■■■■■■■□
Aw, see how you are. I came back to this tab and saw a DB error, so I just hit refresh
-
Options
UnixGuy
Mod Posts: 4,564 Mod
I always recommend RAID 10 + Hot Spare for our customers. Best performance and highest protection. Try to convince your company, tell them that it gives best performance and best protection and once setup, you practically forget about it (less support). -
OptionsTyrant1919
Member Posts: 519 ■■■□□□□□□□
I agree, 10 rocks.A+/N+/S+/L+/Svr+
MCSA:03/08/12/16 MCSE:03s/EA08/Core Infra
CCNA -
Optionsastorrs
Member Posts: 3,139 ■■■■■■□□□□
RAID 10 makes sense - when it makes business sense - it can also lead to costly overprovisioning; everything has its place, hence why I asked for more details.
