This post is an attempt to guide those of you who want to try out linux RAID for performance or other reasons. I have not yet found a suitable GUI tool to do this quickly or simply.
I will not be discussiing "fakeRAID" aka BIOS RAID or hardware RAID controllers. This "how to" strictly linux RAID aka "softRAID" or software RAID. The one major advantage to linux software RAID over these other methods of RAID is if your computer should "die" and need replacement, you can install your hard drives into another computer, start your RAID array, and access your data. If you're using one of the other types and your hardware RAID controller or your motherboard dies - well, good luck!
Your skills before undertaking this adventure should include an understanding of linux partitioning and formatting and how to manually edit text files. You can have a RAID partition running in just a few minutes (or less). I strongly suggest BACKING UP any data you don't want to lose before beginning and use your first attempt as a test, rather than immediately relying on it for primary data storage.
What is RAID? : Combining multiple partitions from two or more hard drives into a single RAID device that can be accessed as a single partition.
Within this post I will also use the terms:
"array" - a RAID device created from two or more partitions
"member" - a partition used within a RAID array
"level" - A number that designates the type of RAID scheme used on a particular array
RAID levels: Rather than go into an indepth discussion here, I will only refer to RAID levels as:
RAID0 aka "striping" - all data is spread across all space without parity checking. Fastest but least secure.
RAID1 aka "mirroring" - all data is duplicated on every partition. No performance gains, but most secure.
RAID5 aka "striping with parity" - all data is distributed across all partitions but some space is used for parity checking. More secure than RAID0 but not as fast. RAID5 requires at least three members.
There are several other RAID levels. Additionally, you can have "nested" arrays. RAID1 on top of RAID0 or similar setups. These topics are too advanced for this discussion. Please search the internet if you would like more information.
Preparation:
If you haven't already - install the package "mdadm".
At least two available partitions on separate hard drives are required for levels 0 and 1. Level 5 requires at least 3 members.
For performance reasons and to prevent wasting hard drive space, all partitions should be nearly or exactly the same size.
Unless otherwise stated: All commands below are in bold typeface and need to be executed in a terminal or console window and require super user access. I will assume you are either using "sudo" with every command or are using a root terminal session. For these examples I will refer to my partitions as /dev/sda5, /dev/sdb5, and dev/sdc5. Obviously, you will use the correct device names for your devices and may also use as many members as you see fit or have available. We will also assume my three partitions are all 5gb in size.
The quick path to your first array:
mdadm --create /dev/md0 --level=0 --raid-devices=3 /dev/sda5 /dev/sdb5 /dev/sdc5
You have just created your first level 0 array! Depending on the level, size and number of members you use it may take a few moments to start up. There are more options when creating - see "man mdadm" for their usage.
To check to see if this array is functioning (sudo not required for this command):
cat /proc/mdstat
results will look something like:
md0 : active raid0 sda5[0] sdb5[1] sdc5[2]
5271328 blocks 64k chunks
Now to use it, treat it just like any other partition, i.e. create a mount point, format it, mount it, put some files on it. Wasn't that simple? NOTE regarding format selection: There is a bug in EXT4 that prevents the use of an entire hard drive as a RAID array if you use the EXT4 format. If you run into this bug, try reducing the size of your members as to leave a small amount of trailing space unused or use EXT3, reiserfs, or some other format more stable than EXT4.
Accessing this array will generally be 3x faster than accessing a single partition and the total array size will be 15gb (3x5gb). Level 5 would result in 10gb of usable space and be about 2x faster and level 1 would have 5gb of space with no increase in speed.
Why use levels 5 or 1?
If one member in a level 0 array fails - all data is lost, period. With level 5, you can replace the failed member and rebuild the array thus recovering most if not all data. Level 1 duplicates all data so as long as you have one member remaining, no data is lost. The level you choose should be determined by the type of use. In my current setup, I use levels 0 and 5. Level 0 for my install partition and for a separate /tmp partition. A level 5 array holds my /home. I forgo the use of level 1 because I am more interested in performance than I require mission critical access to my computer. I also backup important data onto a different computer.
A note about swap partitions:
If you have multiple swap partitions, the correct way to have linux use RAID-like access to them is to assign equal priority during the mount process. It is not recommended that you use a RAID array for swap space. Rather, edit your fstab and add the option pri=1 to all swap mount lines.
At this point, you have a working array but when you re-boot it isn't available. Fear not - the array and it's data remain. You must now "assemble" the device. This can be done in several ways, here's the most basic:
mdadm --assemble /dev/md0 /dev/sda5 /dev/sdb5 /dev/sdc5
You can also use the UUID's of your members (which should all be identical);
mdadm --assemble /dev/md0 --uuid=ee94aa9a:d7763672:577442e6:f4e07f49
NOTE: use the UUID of the individual members, not the array - which will have a unique UUID.
One more method of assembly is to allow mdadm to scan and assemble all your arrays:
mdadm --assemble --scan
As you might guess, it is possible to have your arrays detected and assembled at boot time. This requires your arrays to be listed in /etc/mdadm/mdadm.conf. This automagical way to do this is:
mdadm --examine --scan > /etc/mdadm/mdadm.conf
This will replace any existing mdadm.conf file with the information from all currently running arrays. To amend this file rather than replace it use:
mdadm --examine --scan |grep md0 >> /etc/mdadm/mdadm.conf
This will amend an existing mdadm.conf file with your newly created array data. BTW, the above commands output and mdadm.conf data should look like:
ARRAY /dev/md0 level=raid0 num-devices=3 UUID=ee94aa9a:d7763672:577442e6:f4e07f49
one line for each array and the UUID is again the UUID of the members, not the array.
You can now edit your fstab and have your array mounted at boot, ready for use. Be sure and use the device name or UUID for your array in the mount command - not the individual member device names or UUIDs.
Steps to use a RAID array for a new install: Using a RAID array is the cheapest way to speed up access to your system files and data. I will explain the simplest path to installing Kubuntu to a RAID array. There are other ways, but this method requires the least expertise.
Steps:
1) Create the array(s) before the install: Either boot to a liveCD or USB and install and use mdadm as outlined above.
2) Create a 100mb (thats megabytes, not gigabytes) partition to hold /boot. You cannot boot from a mdadm level 0 or 5 array directly. It is possible to boot to a level 1 array, but that's an advanced topic.
3) Do your new install from the Alternate Install CD: This text-based version of the installer has mdadm support included and will recognize your functioning arrays during partition setup. Remember to use the small partition created in step 2 for /boot or you won;t be able to start your new install.
In my experience with Kubuntu since version 9.04, all created arrays have been detected and used without fail by the Alternate Install disks. There is a menu in the alternate installer during partition setup that allows you to create an array at install time. I have not tried this function.
COMING SOON: a section on removing arrays and prettier formatting of the above - for now, I have to go to work!!!
I will not be discussiing "fakeRAID" aka BIOS RAID or hardware RAID controllers. This "how to" strictly linux RAID aka "softRAID" or software RAID. The one major advantage to linux software RAID over these other methods of RAID is if your computer should "die" and need replacement, you can install your hard drives into another computer, start your RAID array, and access your data. If you're using one of the other types and your hardware RAID controller or your motherboard dies - well, good luck!
Your skills before undertaking this adventure should include an understanding of linux partitioning and formatting and how to manually edit text files. You can have a RAID partition running in just a few minutes (or less). I strongly suggest BACKING UP any data you don't want to lose before beginning and use your first attempt as a test, rather than immediately relying on it for primary data storage.
What is RAID? : Combining multiple partitions from two or more hard drives into a single RAID device that can be accessed as a single partition.
Within this post I will also use the terms:
"array" - a RAID device created from two or more partitions
"member" - a partition used within a RAID array
"level" - A number that designates the type of RAID scheme used on a particular array
RAID levels: Rather than go into an indepth discussion here, I will only refer to RAID levels as:
RAID0 aka "striping" - all data is spread across all space without parity checking. Fastest but least secure.
RAID1 aka "mirroring" - all data is duplicated on every partition. No performance gains, but most secure.
RAID5 aka "striping with parity" - all data is distributed across all partitions but some space is used for parity checking. More secure than RAID0 but not as fast. RAID5 requires at least three members.
There are several other RAID levels. Additionally, you can have "nested" arrays. RAID1 on top of RAID0 or similar setups. These topics are too advanced for this discussion. Please search the internet if you would like more information.
Preparation:
If you haven't already - install the package "mdadm".
At least two available partitions on separate hard drives are required for levels 0 and 1. Level 5 requires at least 3 members.
For performance reasons and to prevent wasting hard drive space, all partitions should be nearly or exactly the same size.
Unless otherwise stated: All commands below are in bold typeface and need to be executed in a terminal or console window and require super user access. I will assume you are either using "sudo" with every command or are using a root terminal session. For these examples I will refer to my partitions as /dev/sda5, /dev/sdb5, and dev/sdc5. Obviously, you will use the correct device names for your devices and may also use as many members as you see fit or have available. We will also assume my three partitions are all 5gb in size.
The quick path to your first array:
mdadm --create /dev/md0 --level=0 --raid-devices=3 /dev/sda5 /dev/sdb5 /dev/sdc5
You have just created your first level 0 array! Depending on the level, size and number of members you use it may take a few moments to start up. There are more options when creating - see "man mdadm" for their usage.
To check to see if this array is functioning (sudo not required for this command):
cat /proc/mdstat
results will look something like:
md0 : active raid0 sda5[0] sdb5[1] sdc5[2]
5271328 blocks 64k chunks
Now to use it, treat it just like any other partition, i.e. create a mount point, format it, mount it, put some files on it. Wasn't that simple? NOTE regarding format selection: There is a bug in EXT4 that prevents the use of an entire hard drive as a RAID array if you use the EXT4 format. If you run into this bug, try reducing the size of your members as to leave a small amount of trailing space unused or use EXT3, reiserfs, or some other format more stable than EXT4.
Accessing this array will generally be 3x faster than accessing a single partition and the total array size will be 15gb (3x5gb). Level 5 would result in 10gb of usable space and be about 2x faster and level 1 would have 5gb of space with no increase in speed.
Why use levels 5 or 1?
If one member in a level 0 array fails - all data is lost, period. With level 5, you can replace the failed member and rebuild the array thus recovering most if not all data. Level 1 duplicates all data so as long as you have one member remaining, no data is lost. The level you choose should be determined by the type of use. In my current setup, I use levels 0 and 5. Level 0 for my install partition and for a separate /tmp partition. A level 5 array holds my /home. I forgo the use of level 1 because I am more interested in performance than I require mission critical access to my computer. I also backup important data onto a different computer.
A note about swap partitions:
If you have multiple swap partitions, the correct way to have linux use RAID-like access to them is to assign equal priority during the mount process. It is not recommended that you use a RAID array for swap space. Rather, edit your fstab and add the option pri=1 to all swap mount lines.
At this point, you have a working array but when you re-boot it isn't available. Fear not - the array and it's data remain. You must now "assemble" the device. This can be done in several ways, here's the most basic:
mdadm --assemble /dev/md0 /dev/sda5 /dev/sdb5 /dev/sdc5
You can also use the UUID's of your members (which should all be identical);
mdadm --assemble /dev/md0 --uuid=ee94aa9a:d7763672:577442e6:f4e07f49
NOTE: use the UUID of the individual members, not the array - which will have a unique UUID.
One more method of assembly is to allow mdadm to scan and assemble all your arrays:
mdadm --assemble --scan
As you might guess, it is possible to have your arrays detected and assembled at boot time. This requires your arrays to be listed in /etc/mdadm/mdadm.conf. This automagical way to do this is:
mdadm --examine --scan > /etc/mdadm/mdadm.conf
This will replace any existing mdadm.conf file with the information from all currently running arrays. To amend this file rather than replace it use:
mdadm --examine --scan |grep md0 >> /etc/mdadm/mdadm.conf
This will amend an existing mdadm.conf file with your newly created array data. BTW, the above commands output and mdadm.conf data should look like:
ARRAY /dev/md0 level=raid0 num-devices=3 UUID=ee94aa9a:d7763672:577442e6:f4e07f49
one line for each array and the UUID is again the UUID of the members, not the array.
You can now edit your fstab and have your array mounted at boot, ready for use. Be sure and use the device name or UUID for your array in the mount command - not the individual member device names or UUIDs.
Steps to use a RAID array for a new install: Using a RAID array is the cheapest way to speed up access to your system files and data. I will explain the simplest path to installing Kubuntu to a RAID array. There are other ways, but this method requires the least expertise.
Steps:
1) Create the array(s) before the install: Either boot to a liveCD or USB and install and use mdadm as outlined above.
2) Create a 100mb (thats megabytes, not gigabytes) partition to hold /boot. You cannot boot from a mdadm level 0 or 5 array directly. It is possible to boot to a level 1 array, but that's an advanced topic.
3) Do your new install from the Alternate Install CD: This text-based version of the installer has mdadm support included and will recognize your functioning arrays during partition setup. Remember to use the small partition created in step 2 for /boot or you won;t be able to start your new install.
In my experience with Kubuntu since version 9.04, all created arrays have been detected and used without fail by the Alternate Install disks. There is a menu in the alternate installer during partition setup that allows you to create an array at install time. I have not tried this function.
COMING SOON: a section on removing arrays and prettier formatting of the above - for now, I have to go to work!!!
Comment