PDA

View Full Version : Utility of Raid 5



zano65
2006-03-23, 09:12
Hello, i've heard of Raid 5, if someone is using it i would like to have an opinion. Are the disks seen as one disk? It would be easier to browse with the Squeezebox. Is it faster and reliable?
I would like to buy a Pci SATA card and 4 300 G Hdds in the future.

Thanks in advance.
Jean

moby_uk
2006-03-23, 09:25
Raid 5 is great. I have 4x250Gb disks of which 750Gb is useable. I can lose any single disk without data loss. If it fails, I swap the faulty disk with one of equal or greater capacity and it automatically rebuilds (in theory). One nice feature as well is that I can continue to use the system - so when I installed it first time, I was installing the operating systems whilst the whole array was building itself.

Configuration is vendor dependant, but on mine (an adaptec 2140SA), you add disks to an array, which then appears as a logical disk. So for me, my four disks are presented as two logical drives, which I can then partition as I like (as shown in the extensive diagram below)

Physical disks -> Array -> Logical disks -> Partitions

In practice it works very well, in that there is no difference between this and stand-alone disks.

jimdibb
2006-03-23, 09:29
In general (there are lots of different devices so specifics may differ) all
the disks in a Raid 5 are seen as a single logical disk. Some versions let
you configure the set of disks as multiple logical disks.

For Read-Mostly applications like the squeezebox/slimserver, the performance
is generally N times better than a single disk where N +1 is the total
number of disks you have (3 times faster for your 4 total drives
mentioned). Write performance varies based on the exact pattern of
accesses.

It is much more reliable than a single disk, but for most purposes you
should still have backups. (music is generally not critical, but it is
expensive/time consuming to replace)

On 3/23/06, zano65 <zano65.254riz1143130501 (AT) no-mx (DOT) forums.slimdevices.com>
wrote:
>
>
> Hello, i've heard of Raid 5, if someone is using it i would like to have
> an opinion. Are the disks seen as one disk? It would be easier to
> browse with the Squeezebox. Is it faster and reliable?
> I would like to buy a Pci SATA card and 4 300 G Hdds in the future.
>
> Thanks in advance.
> Jean
>
>
> --
> zano65
> ------------------------------------------------------------------------
> zano65's Profile: http://forums.slimdevices.com/member.php?userid=3906
> View this thread: http://forums.slimdevices.com/showthread.php?t=22379
>
>

kkitts
2006-03-23, 11:13
It is much more reliable than a single disk, but for most purposes you
should still have backups.
>
> [/color]

The age-old question for me is how one realy backs up disk arrays this large. Currently, I have a very simple setup a 320G drive in the slimserver. And another machine on another level of the house (connected via 100mbps wired etheret) that I use with a 300G drive to do backups of the slimserver. I have no tape/cd/dvd backups - just not enough time to make those...

Kevin

bklaas
2006-03-23, 11:22
If you are on linux I recommend you take a look at http://rsnapshot.org (discussed on the 'how do you backup...' thread of a few days ago as well)

This would be a perfect solution for what you just described, kkitts.

Also, FWIW, I'd recommend your backup drive size >= primary drive size. rsnapshot can save multiple, full snapshots of your primary drive with a very minimal amount more space used, but it'll still be bigger than your primary HD.

cheers,
#!/ben

BillC
2006-03-23, 11:26
The age-old question for me is how one realy backs up disk arrays this large.

Yep. I was going to ask about that very thing. What, if anything, are people using to back up a large music library. I have my ripped files on a RAID5 NAS. Backing up to DVD would be a daunting task. The only real solution seems to be occasionally taking a snapshot onto another drive - which calls into question the whole idea of RAID 5 versus RAID 1 in the first place.

JJZolx
2006-03-23, 12:29
Hello, i've heard of Raid 5, if someone is using it i would like to have an opinion. Are the disks seen as one disk? It would be easier to browse with the Squeezebox. Is it faster and reliable?
I would like to buy a Pci SATA card and 4 300 G Hdds in the future.
It has some advantages.

The single biggest reason to use RAID is that it allows you to keep running with _no_down_time_ in the event of a disk failure. Note the "no down time". That's key to business systems and others that must run 365x24x7. Probably not so important to a home media server. If you can afford a few hours of down time and can restore the data on a failed drive from a backup, then a non-RAID storage solution will work just about as well.

Yes, you can create a single volume across all of the drives in the array, which is nice. This is a convenient way to create very large volumes. But it comes at a price.

Reading from a RAID 5 array will generally be faster than from a standalone drive. The more disks you have in the arrary, the faster it should be. Limitations in RAID controller speed, bus speed, and even CPU speed may limit this, however, depending on the particular installation.

Writing is a bit slower, since parity has to be computed. A good RAID controller will minimize the write penalty.

There are also disadvantages.

You lose the capacity equivalent to one drive in the array for the parity information. But the more drives in the array, the less the penalty. The smallest RAID 5 array you can build would be across three disks, meaning you'd pay a 33% premium over using those disks by themselves. In a four disk array, it would be 25%, etc.

If your RAID controller card fails, you will lose the array. Usually you can replace the controller with another identical one and you'll be back and running, but this can be tricky.

Managing the array itself can be tricky. If a disk dies, you need to know about it ASAP, then you need to make sure you replace the correct disk or you'll wipe out the array. Good software and a good controller card are your only hope.

jimdibb
2006-03-23, 13:45
Other disks (bigger, slower) is the primary way at the moment, unless you've
done it incrementally over time to smaller media (backing up your library to
DVD as you rip it will make the ripping take longer but is much less
daunting than doing the same thing after ripping the whole thing)

In dealing with disk arrays that hold 60+ TB for commercial use, backup to
disk is also popular. The product I'm most familiar with is a device that
emulates a tape drive interface (so backup software is comfortable) but uses
a RAID array as the storage (with bigger, cheaper, slower, disks than the
primary on-line array its backing up). It also then has a transparent way
to copy the disk-based tapes to real tapes for offline backup.

Mirroring to another site -- either synchronously or asynchronously -- (like
many here do) is also very popular. It's just on a much bigger scale.

For R5 vs. R1, R5 is cheaper and the real reliability is only slightly worse
than mirrors (gets worse with wider Raid groups, but also gets cheaper). R1
needs back-ups just as much as R5 does.

On 3/23/06, BillC <BillC.254xrz1143138601 (AT) no-mx (DOT) forums.slimdevices.com>
wrote:
>
>
> kkitts Wrote:
> > The age-old question for me is how one realy backs up disk arrays this
> > large.
>
> Yep. I was going to ask about that very thing. What, if anything, are
> people using to back up a large music library. I have my ripped files
> on a RAID5 NAS. Backing up to DVD would be a daunting task. The only
> real solution seems to be occasionally taking a snapshot onto another
> drive - which calls into question the whole idea of RAID 5 versus RAID
> 1 in the first place.
>
>
> --
> BillC
> ------------------------------------------------------------------------
> BillC's Profile: http://forums.slimdevices.com/member.php?userid=1235
> View this thread: http://forums.slimdevices.com/showthread.php?t=22379
>
>

BillC
2006-03-23, 14:39
For R5 vs. R1, R5 is cheaper and the real reliability is only slightly worse
than mirrors (gets worse with wider Raid groups, but also gets cheaper). R1
needs back-ups just as much as R5 does.

In the general case I'll agree that R1 needs backups as much as R5. In the case of a music library the chance of a write corruption that propogates across the mirrors is pretty low since almost all access is read only. In this case I think R1 is pretty close functionally to a backup. But as you said, with only a slight reliability advantage and a cost disadvantage compared to R5 - which is why I'm using R5 in the first place.

I guess I'll just keep on with my R5 array and my "back-up" the physical media that I've ripped.

cjhabs
2006-03-23, 14:48
It's time to pull out the wallet and spring for RAID-10 :-)

jimdibb
2006-03-23, 14:56
I wasn't referring to write corruption but the chance of simultaneous loss
of 2 drives. If you have a drive failure in your R1, it still needs to be
replaced and rebuilt before the second drive also fails.

2 drive failure is also what you primarily worry about in R5 systems, but
the chance of two drives failing is larger on R5 because you have more
drives that could possibly fail.

Once the first drive fails on a R1, the second drive must fail too.
Once the first drive fails on R5, one of N-1 drives must also fail.
If the reliability of any drive is .99 for a given time frame, the
reliability of N of them is (.99)^N. So the second drive in a mirror is .99
reliable. The set of 3 other drives in a R5 is (.99)^3 or .97.

On 3/23/06, BillC <BillC.2556ko1143150002 (AT) no-mx (DOT) forums.slimdevices.com>
wrote:
>
>
> jimdibb Wrote:
> >
> > For R5 vs. R1, R5 is cheaper and the real reliability is only slightly
> > worse
> > than mirrors (gets worse with wider Raid groups, but also gets
> > cheaper). R1
> > needs back-ups just as much as R5 does.
> >
> In the general case I'll agree that R1 needs backups as much as R5. In
> the case of a music library the chance of a write corruption that
> propogates across the mirrors is pretty low since almost all access is
> read only. In this case I think R1 is pretty close functionally to a
> backup. But as you said, with only a slight reliability advantage and a
> cost disadvantage compared to R5 - which is why I'm using R5 in the
> first place.
>
> I guess I'll just keep on with my R5 array and my "back-up" the
> physical media that I've ripped.
>
>
> --
> BillC
> ------------------------------------------------------------------------
> BillC's Profile: http://forums.slimdevices.com/member.php?userid=1235
> View this thread: http://forums.slimdevices.com/showthread.php?t=22379
>
>

jimdibb
2006-03-23, 15:00
Smiley not withstanding,

R10 is neither more reliable nor cheaper than R1. Nor more expensive
really. Also, the math from my previous post applies for why it's at all
more reliable than R5.

Just keeping it real. :)

On 3/23/06, cjhabs <cjhabs.25571b1143150601 (AT) no-mx (DOT) forums.slimdevices.com>
wrote:
>
>
> It's time to pull out the wallet and spring for RAID-10 :-)
>
>
> --
> cjhabs
> ------------------------------------------------------------------------
> cjhabs's Profile: http://forums.slimdevices.com/member.php?userid=3760
> View this thread: http://forums.slimdevices.com/showthread.php?t=22379
>
>

Robin Bowes
2006-03-23, 15:44
Jim Dibb wrote:
> Smiley not withstanding,
>
> R10 is neither more reliable nor cheaper than R1. Nor more expensive
> really. Also, the math from my previous post applies for why it's at
> all more reliable than R5.

Actually, it's slightly more reliable. It can withstand the loss of 2
discs (one in each mirror) whereas R1 can only survive the lost of one disc.

R.

Robin Bowes
2006-03-23, 15:48
Jim Dibb wrote:
> I wasn't referring to write corruption but the chance of simultaneous
> loss of 2 drives. If you have a drive failure in your R1, it still
> needs to be replaced and rebuilt before the second drive also fails.
>
> 2 drive failure is also what you primarily worry about in R5 systems,
> but the chance of two drives failing is larger on R5 because you have
> more drives that could possibly fail.
>
> Once the first drive fails on a R1, the second drive must fail too.
> Once the first drive fails on R5, one of N-1 drives must also fail.
> If the reliability of any drive is .99 for a given time frame, the
> reliability of N of them is (.99)^N. So the second drive in a mirror is
> .99 reliable. The set of 3 other drives in a R5 is (.99)^3 or .97.

....and because all the drives are often around the same age and
operating in the same environment, the other disks in a R5 array can be
"ready to fail" too and the additional load/stress of resyncing after a
drive failure can be enough to tip them over the edge.

Personally, I have a 6-disk R5 array - 4 + 1 parity + 1 hot spare

I'll be going with RAID6 next time - that can withstand the loss of two
drives.

R.

jimdibb
2006-03-23, 16:18
The same applies to the mirror. The surviving drive is under just as much
stress during resynching.

On 3/23/06, Robin Bowes <robin-lists (AT) robinbowes (DOT) com> wrote:
>
> Jim Dibb wrote:
> > I wasn't referring to write corruption but the chance of simultaneous
> > loss of 2 drives. If you have a drive failure in your R1, it still
> > needs to be replaced and rebuilt before the second drive also fails.
> >
> > 2 drive failure is also what you primarily worry about in R5 systems,
> > but the chance of two drives failing is larger on R5 because you have
> > more drives that could possibly fail.
> >
> > Once the first drive fails on a R1, the second drive must fail too.
> > Once the first drive fails on R5, one of N-1 drives must also fail.
> > If the reliability of any drive is .99 for a given time frame, the
> > reliability of N of them is (.99)^N. So the second drive in a mirror is
> > .99 reliable. The set of 3 other drives in a R5 is (.99)^3 or .97.
>
> ...and because all the drives are often around the same age and
> operating in the same environment, the other disks in a R5 array can be
> "ready to fail" too and the additional load/stress of resyncing after a
> drive failure can be enough to tip them over the edge.
>
> Personally, I have a 6-disk R5 array - 4 + 1 parity + 1 hot spare
>
> I'll be going with RAID6 next time - that can withstand the loss of two
> drives.
>
> R.
>
>

jimdibb
2006-03-23, 16:18
The reliability is only per protected set. Which in a R10 still is only 2
drives. Consider per usable capacity.

On 3/23/06, Robin Bowes <robin-lists (AT) robinbowes (DOT) com> wrote:
>
> Jim Dibb wrote:
> > Smiley not withstanding,
> >
> > R10 is neither more reliable nor cheaper than R1. Nor more expensive
> > really. Also, the math from my previous post applies for why it's at
> > all more reliable than R5.
>
> Actually, it's slightly more reliable. It can withstand the loss of 2
> discs (one in each mirror) whereas R1 can only survive the lost of one
> disc.
>
> R.
>
>

gharris999
2006-03-23, 19:59
If you are working in a Windows XP environment, you can play around with software raid 5 by following the recipe here: http://www.tomshardware.com/2004/11/19/using_windowsxp_to_make_raid_5_happen/index.html

This does involve taking a hex editor to a couple of windows system files, but I've had very good luck with this setup. My software raid 5 array easily survived a system reinstall, which is more than you could say for the software raid that was built into WinNT 4 Server.

Eventually, I hope to migrate my slimserver box to some flavor of linux and I understand that there is good software raid support there too.

dean
2006-03-23, 20:56
My experience with raid5 was a bad one. I had one drive fail on my
linux server and didn't get notified for a long time. Then when I
went to add another drive and rebuild, another (of the same vintage)
went out.

Now, I'm not 100% sure that it wasn't my fumbly fingers that did
something to corrupt the drives during reconstruction, but it was a
lesson learned.

That was with 80G drives. When I went to rebuild, I bought 2 250G
drives and have a nightly rsync to back up one to the other. Plus I
have another drive in another machine with a more infrequent backup
from the main set in case the box catches fire or something. And
another one in my brother's house that gets backed up even more
infrequently, in case the house catches fire or something.

If my main drive goes out, I'm at risk of losing up to 24 hours worth
of changes. That's ok by me.

And, if my fumbly fingers do something stupid, I don't end up wiping
out both copies.

So, my experience is that you need to consider hardware, software and
wetware failures when protecting your data.

Rene
2006-03-24, 02:10
Well, I have had only positive expierience with raid5. I have been using it for several years now. Lost one controller about 2 years ago and a couple of drives in the past 3 years. No problems to restore the system in all cases. Using good raid controllers also helps, as they inform you about problems ahead of failure.
My current music archive setup is 2x raid5 systems with 4TB each, rsynched. This gives me enough protection against hardware failure until we can get DVS's holding 1TB :-)
regards
Rene

=?ISO-8859-1?Q?Peter_N=F5u?=
2006-03-24, 02:22
Robin Bowes wrote:
>
> Personally, I have a 6-disk R5 array - 4 + 1 parity + 1 hot spare
>
> I'll be going with RAID6 next time - that can withstand the loss of two
> drives.

and what controller might you invest in for this (future) setup?
something relatively cheap that can be installed in old hardware?

/peter

Robin Bowes
2006-03-24, 06:56
Peter Nu wrote:
> Robin Bowes wrote:
>
>>
>> Personally, I have a 6-disk R5 array - 4 + 1 parity + 1 hot spare
>>
>> I'll be going with RAID6 next time - that can withstand the loss of two
>> drives.
>
>
> and what controller might you invest in for this (future) setup?
> something relatively cheap that can be installed in old hardware?

I'm currently using 2 x Promise SATA150-TX4 4-port SATA controllers in
an old 1Ghz PIII mobo, with three disks each controller.

R.

stinkingpig
2006-03-24, 07:40
dean blackketter wrote:
> My experience with raid5 was a bad one. I had one drive fail on my
> linux server and didn't get notified for a long time. Then when I
> went to add another drive and rebuild, another (of the same vintage)
> went out.
>
> Now, I'm not 100% sure that it wasn't my fumbly fingers that did
> something to corrupt the drives during reconstruction, but it was a
> lesson learned.

probably no mistake. One of the most common Nagios monitors I've
provided for people is a Linux software RAID monitor. Naturally I
haven't kept a copy, but it's pretty simple, just awk /proc/mdstat and
alert if the number of good drives changes.

--
Jack at Monkeynoodle dot Org: It's a Scientific Venture...
Riding the Emergency Third Rail Power Trip Since 1996

Robin Bowes
2006-03-24, 13:12
Jack Coates wrote:
> dean blackketter wrote:
>
>> My experience with raid5 was a bad one. I had one drive fail on my
>> linux server and didn't get notified for a long time. Then when I
>> went to add another drive and rebuild, another (of the same vintage)
>> went out.
>>
>> Now, I'm not 100% sure that it wasn't my fumbly fingers that did
>> something to corrupt the drives during reconstruction, but it was a
>> lesson learned.
>
>
> probably no mistake. One of the most common Nagios monitors I've
> provided for people is a Linux software RAID monitor. Naturally I
> haven't kept a copy, but it's pretty simple, just awk /proc/mdstat and
> alert if the number of good drives changes.

mdadm has a monitor mode that does what it says on the can - monitors md
devices.

Indeed, Redhat/Fedora comes with a service (mdmonitor) that uses mdadm
--monitor to do just that.

R.