One For The Gods Of Linux Partitions / Raid / LVM / (REDHAT)

Hey guys looking at trying to get my head around partitions I am going to keep it simple to build a foundation.
Currently I have two identical hard disks both are 250GB so Dev/sda and Dev/sdb

From a clean install perspective what is the best way of setting up these drives? I wanted to use LVM for future
expandability and RAID for redundancy. I want there to be a /data partition as well for storage.

Am I right in saying that you setup LVM first and then add the Raid Level? I am going to be using Centos.

If anyone has a easy to follow Redhat based tutorial that would be amazing or could explain the process
in setting up the LVM that would be kool I understand the concept but not how to implement it from a clean install
situation.

Find more posts tagged with

Comments

W Stewart

Set up the RAID array first and then turn the array (/dev/md0) into an lvm physical volume. Then create your volume group from the physical volume and lvm from the volume group.

As far as the partitions, you can set them up however you want. LVM does give you a lot of flexibility to resize things if you know what you're doing. I typically have a separate home partition in case I decide to install another distro so I don't lose any of my personalized configuration although I usually have to delete some gconf files so things don't break. You'll need a separate boot partition if you're putting the / partition on an lvm and I believe with mdadm, you have to actually install grub, to the mbr of both individual drives. Not sure what you need a separate /data or storage partition for but I usually have an extra data partition with music on it that I share between my linux and windows OS(dual boot). Just do the math and figure out how much space you want for each partition ahead of time.

I believe the CentOS installer may have graphical tools to allow you to setup RAID and I'm pretty sure they allow you to configure LVMs as well but if you run into trouble you could always set up your partitions from a live cd prior to installing. Here's a link that should get you started in the right direction.

3TB HDD + RAID5 + LVM on CentOS 6.2 « Richard's Blog

The Technomancer

I'm not a god, but I am a mother@#$%in' sorceror!

Unless you're doing this as an exercise to familiarize yourself with LVM, don't bother splitting your RAID into /, /home, /var, /data, etc. It's all on the same logical disk, and it's a pain in the ass to non-destructively shrink an LVM volume if you need more space later on one of the smaller volumes. If you're going to be using some of the fancier RAID features like a fully named array or some of the newer metadata options, then sure, but otherwise, there's a real chasm between the amount of work that needs to be done for any performance rewards and the amount of performance you're rewarded with.

First rule of Linux systems administration -- just because you can do something kickass and cool, doesn't mean that it's supportable in a live environment or supportable by the poor sap that comes along after you. This is also why you should always comment your code like the person that'll be maintaining it after you is a psychopathic murder that knows where you live.

But hey, since we're here, we'll do it anyway. Just know that if this system is going to be up for any length of time and actually has a chance at using up some of the smaller partitions, you're going to hate your life, especially if its / you need to pull space from or add space to later on down the road.

Anyway, assuming our disks are /dev/sdX:

mdadm --create /dev/md0 --level=<raid level number> --raid-devices=<number of devices> <list of devices>

So if we just want a simple two-disk RAID 1 from /dev/sda and /dev/sdb, which are 1TB disks:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda /dev/sdb

Toss --verbose in there if you want the OS to tell you what exactly is going on. Same thing, but with a spare disk just in case..

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda /dev/sdb --spare-devices=1 /dev/sdc

Check /proc/mdstat, and you should see that /dev/md0 has been started.

Now you have a RAID1. Note down the chunk size (or do one better and set it to 32K or something like that while creating the RAID, I'll let you look that one up), you'll need that for the LVM portion. Default is 64K, which you get from the above commands.

Now, we register/create that physical volume with LVM using pvcreate:

pvcreate -M2 --dataalignment 64K /dev/md0

That creates a physical LVM2 (-M2) volume with data aligned on the chunk size of 64k using dev md0.

If your chunk size is something other than 64K, put that there instead.

Now we have a physical volume registered (and you can check this with pvdisplay) -- time to create a volume group on it that we can make our LVMs in.

vgcreate NameThisVG00 -s 32M /dev/md0

Now we've created a VG named "NameThisVG00" with a 32M extent size (and this value should always be both a power of 2 and a multiple of your chunk size) on /dev/md0. Verify using vgdisplay, and start making volumes!

Since we've got around 1TB to play with, we'll go with a 50GB /, 200GB /home, and the remainder as /data.

lvcreate -L 50G -n 'lv00_root' NameThisVG00
lvcreate -L 200G -n 'lv01_home' NameThisVG00
vgdisplay NameThisVG00 (to see the remaining free extents)
lvcreate -l <number of free extents> -n 'lv02_data' NameThisVG00

Voila! You now have three logical volumes. Now, put your filesystem of choice on them (And label your disks when doing so, makes managing /etc/fstab so much easier), add the disks to /etc/fstab, and mount away!

--

Questions...

But Technomancer, you forgot swap space and this is going to be my main filesystem!

Swap is evil and saps performance. But if you really need it, make a swapfile using dd, format it as swap space, and use swapon to use it. You're writing to the same disk anyway -- save yourself the hassle of having another partition to manage.

How the hell do I do all that, Technomancer?!

Make a thread and ask! It's 1:14am here prior to Thanksgiving, so this is all you're getting tonight.

But what if I want to do something crazy like multiple LVs across multiple disks, or offset my VGs, etc...

If you actually have a good reason to do this, someone would have told you to do it. If they did tell you to do it and you hit this post via a Google search, my consulting rates are $150/hr. PM me!

But Technomancer, can't I use a GUI tool for this?

What are you, a Windows admin? Sure, you can. But you will never, ever, ever, ever, ever have a GUI in a live Linux environment, and if you do, find a new job because the place you're working at is doing it wrong.

dpsmooth15

this guy is awesome (Technom). You make sure NOBODY can even attempt to reply with a better answer after you, and make the guy that replied before you feel like ……

ally_uk

The Technomancer is amazing

It's like having your own personal Linux Guru, fantastic explanation as always really clear and concise. I was just using the whole exercise as way of familiarizing myself with both LVM and Raid.

Just going to throw in another question when creating the RAID array how do you do this during the install? do you have to bypass the graphical / text partition tools and jump into the CLI to manually create your partitions if so how do you do this? alt F2

ally_uk

Sorry to bump this but it was a fantastic explanation but I am still getting a bit confused. During the install process how the hell do I get to enter those fancy raid creation commands? Do I have do run a live cd and partition before I install?

The Technomancer

Yeah. If you're PXE booting, it'll be in your kickstart file. Otherwise, use a live CD.

You'll actually want to make a boot partition though. Make one the same size on both disks, then when you're doing the RAID stuff, use sda2 and sdb2 for the mdadm commands.