PDA

View Full Version : Headless Linux DIY contra turnkey NAS



trebejo
2005-11-26, 19:39
People here seem to be very interested in black-boxing their music collection, and the topic of which NAS device to use keeps coming up here and there. Some folks have given very useful info but it's a bit scattered among many threads.

I am hoping to focus the above on a specific thread, and in particular to broaden the option to include a do-it-yourself scenario. So I'm going to provide some bytes for the savvy among us to chew on.

The reason that I haven't yet plunked down the cash for an Infrant box is that I'm not sure whether I'll be happy with it for reasons having to do with cpu cycles and noise. Since these boxes are not available for review at your local Fry's, you basically have to dive in and buy one before you personally experience it.

Regarding the cpu cycles, I am concerned about how slimserver will run with a ReadyNAS (seeing how it eats cpu cycles in my venerable powerbook even with its 1 GB of RAM, it is something to think about); for example, waiting for several minutes or even hours before new additions to the music library show up ready to play. I have read the occasional complaint about this from ReadyNAS users so it's certainly a source of uncertainty.

Regarding noise issues, if the NAS device is noisy then it defeats the audiophile raison d'etre of the squeezebox in an obvious way. The alternative is to place the NAS device in a separate room or somesuch, and rig up the network (e.g. good cable layout, satisfactory wireless performance) so that it's still accessible. Doable, but in the end a quiet device would certainly be preferable and give us more flexibility.

Apparently the ReadyNAS performs better on both criteria above than the Buffalo and others; however it's a newish market and that may change.

The cost of the ReadyNAS barebones is about $600 plus shipping and taxes. I am certain that for that money one can build a barebone linux box of greater capabilities. It's been a few years since I've put together a linux system (Suse 8) and I've never done the headless thing so it would involve some rtfming. I don't know whether I can use Herger's SlimCD to manage a RAID system as well as run the slimserver; that would be an obvious win wrt what OS to install. If you haven't seen it yet you can read about slimcd at

http://www.herger.net/slim/detail.php?nr=763&kategorie=slim

If you've set up a RAID/Slimserver system using linux and you'd like to comment on what you did wrt software I would appreciate it.

Now onto the hardware side of things. The linux box should definitely have room for four hard drives, since RAID 5 hits a sweet spot at that number. One wants to run RAID 5 because it provides a complete backup of the content of any one drive in the array, and because it does so without consuming so much hard drive space in the process. There are other RAID options and in some situations they are the way to go but for "us" it looks like RAID 5 is the clear choice and four drives is a good ergoecotechnomic minimax fit.

I don't know if the box should have an additional drive to host the OS and so forth. The advantage of that that I see is that one can then take out the drives and put them in some future container should the need arise, without violating the integrity of the past and future host boxen. Perhaps there is some performance advantage as well to having the system on one drive and the data on another (although I doubt that makes an important difference in our case). Since the DIY cases tend to have plenty of room it is certainly feasible to spend an extra $50 and get a 100GB+ hard drive on which to install linux and whatever other OS tickles your fancy. An extra hard drive not only raises the cost, however, it also raises the noise level (more on that below).

RAM is good and cheap and a gigabyte is a nice round number; that's another $100. Motherboard and cpu together will probably go for about $200; I had a look-around at the local Fry's and saw plenty of motherboards for about $100 that have integrated gigabit ethernet (which is helpful), but I haven't considered what it takes to run RAID proper. As for CPUs it's been ages since I looked at the intel side of things (I got off at the Athlon 1.x ghz models) but for a long time it's been the case that you can get a decent "nothing fancy" cpu from amd for at most $100. So on the way to the "barebone" (meaning, before the four RAID drives) we spend about $300 on the motherboard and the stuff that sits right on top of it.

The noise reduction require a refined touch, as one has to travel inside the computer case looking for specific noise sources and then invest time, money and elbow grease to diminish or even eliminate them. I've gone to a couple of web sites to get my thinking started along the "quiet" lines:

http://www.silentpcreview.com/index.php

http://www.endpcnoise.com/

There is a (computer) case that has garnered some nice reviews and it can be obtained for less than $100 (I believe "Radish" pointed it out on these fori):

http://www.silentpcreview.com/article116-page1.html

Other hardware considerations that I've thought about but for which I have little specific knowledge are:

1. Fan quality makes a big difference; apparently one can invest for ~$10 rather than ~$3 fans and lose a lot of noise. But I've seen ads for fans that cost $30+ so I'm still working on this clue.

2. Hard drives HAVE to make noise, and the noisy ones make a lot of it. Specific recommendations for SATA hard drives in the 250GB+ category would be most apropos. There is a case available that surrounds a hard drive and curtails its noise, but I am concerned about the hard drive heating up (even though the merchant claims that that's taken care of in the design of the case):

http://www.endpcnoise.com/cgi-bin/e/driveaway_black.html

3. If the linux box can be made to run without a VGA card, then that's an extra fan that doesn't have to run (not to mention a savings of a few bucks). Motherboards seem to have integrated vga now but I don't know if that means running an extra fan. Running headless is better all around wrt the "black box" approach to feeding music to the squeezebox, anyway.

So adding up the tab, we get motherboard, cpu, RAM, and case for about $400. Presumably this setup works better than, say, the ReadyNAS (albeit in a larger package), but one still has to rtfm and assemble (or hire someone else to do so, probably for another $100 or so).

The hard drives then pop in at one's leisure. The current sweet spot seems to be 400gb (after taking the fixed cost of the box into account), but the 300gb are pretty much in the same ballpark in the $/gb sense so if one wishes to reduce the initial outlay by $200-300 then that option is available. Next year when the 400gbs are cheap (or so I dream) then I can order another drive and keep it in the closet for that fateful day. The 400gb's go for a shade over $200 each so four of them puts the total a bit over $800.

[Aside: when the hard drive is advertised at 400 GB it actually only means 372 GB. The reason is a marketing one, having to do with whether one uses 1000 or 2^10 = 1024 as "a kilobyte". The marketing guys use 1000 but the computer uses 2^10 so you need to divide the stated size by 1.024 to get the actual size).

So $400 + $800 + shipping/taxes brings the total for the dyi to about $1300. That would of course be a monster storage device at around 1.1 terabytes fully backed up, which is enough room for about 3300 CDs compressed in flac format. This would plug in quietly, efficiently and reliably (because we made it so) into your hi-fi rig, and give you storage for your other data needs. Indeed, this would be a case of the hi-fi gadget that actually helps out with all the other aspects of your daily living, which is certainly a reversal of the usual order.

I'll pause now so you can drool a little bit.

To sum up, I'd rather go DIY with the headless linux box for performance and compatibility/long-term reasons (after all Infrant could always get bought out by Jobs or Gates and turned into a crap product for some evil corporate reason). The feared rtfm factor sits there waiting, however; and the box does end up being bigger than the ReadyNAS. And who knows, maybe after I finish the DIY it ends up being noisier too! But I'm pretty sure it would run better... ahem, once the kinks were worked out.

stinkingpig
2005-11-26, 20:35
....
> The reason that I haven't yet plunked down the cash for an Infrant box
> is that I'm not sure whether I'll be happy with it for reasons having
> to do with cpu cycles and noise. Since these boxes are not available
> for review at your local Fry's, you basically have to dive in and buy
> one before you personally experience it.
> ...

noise and speed levels are both highly subjective, edging towards the
audiophile territory in terms of unmeasurable nontransferability. I've
seen lots of posts, articles, and emails claiming that Shuttle SFF's are
quiet enough and Mini-ITX's are fast enough, for instance, which are both
inaccurate claims as far as I'm concerned. If you have doubt before even
plunking down coin, I'd suggest buying a proper desktop computer and
storing it in another room. Furthermore, if you don't want to be
troubleshooting some very, very, very frustrating networking issues, I'd
suggest doing what it takes to get a length of CAT-5 between that computer
and your stereo. Wireless is great when it works, but it works really
poorly in most real-world scenarios.

> If you've set up a RAID/Slimserver system using linux and you'd like to
> comment on what you did wrt software I would appreciate it.
>

I've done both, but not on the same box :) Software RAID sucks because
neither Windows nor Linux will reliably inform you that a drive has failed
or reliably rebuild after that failure. Hardware RAID sucks because the
board is just as likely to fail as the drives, and every board is
incompatible. Whichever one you choose, you will need a backup or else you
will lose data -- RAID only buys data security for the lucky, its real
purpose is to buy uptime.

If I were building a new server box, it would look a lot like my old one
(only with more coin dropped on the hard drives). I'd also go SATA instead
of EIDE.

1) middle-of-the-road Intel or AMD desktop motherboard with built-in VGA
and Ethernet.
2) middle of the road CPU for same
3) as much RAM as the board will hold (can be reduced if you're building a
single purpose box, but I much prefer one relatively quiet box running
everything all the time to lots of boxes needing updates and TLC).
4) two relatively small hard drives for the OS, say 20GB? RAID1 mirror
would be okay, but I'd prefer to put the OS on one and copy it to the
other every night.
5) two honking big hard drives for the media, say 300GB. Again, I'd prefer
to copy every night.

http://www.monkeynoodle.org/comp/tools/backups for a simple "back
everything up" solution; I've been using it successfully for years.

--
Jack At Monkeynoodle.Org: It's A Scientific Venture...
"Believe what you're told; there'd be chaos if everyone thought for
themselves." -- Top Dog hotdog stand, Berkeley, CA

pfarrell
2005-11-26, 21:21
On Sat, 2005-11-26 at 18:39 -0800, trebejo wrote:
> I am hoping to focus the above on a specific thread, and in particular
> to broaden the option to include a do-it-yourself scenario. So I'm
> going to provide some bytes for the savvy among us to chew on.

Before I bought my first SqueezeBox, I was planning on going
with a suitably small box in my main stereo listening room.
Granted, this was years ago, but I decided that the
SqueezeBox approach with a computer in my basement was
the way for me. For many of the reasons you talk about.


> Regarding noise issues, if the NAS device is noisy then it defeats the
> audiophile raison d'etre of the squeezebox in an obvious way.

I don't agree, as I don't see any reason to have the SlimServer box
in the same room as my serious stereo. Its next to the furnace, which
makes lots of noise on its own in the summer and winter.


> The cost of the ReadyNAS barebones is about $600 plus shipping and
> taxes. I am certain that for that money one can build a barebone linux
> box of greater capabilities. It's been a few years since I've put
> together a linux system (Suse 8) and I've never done the headless thing

I think you can get a generic whitebox machine for half or a third of
that. I bought a serious linux box (I write software for a living,
so I wanted very fast) for $400 a couple of months ago. It has
a fast AMD CPU, a gig of Ram, a good video card, case, and a 80 GB disk.
Going to a 300GB disk would have added maybe $50 to it.

I'm not sure what you mean by "headless"
I know you mean "doesn't require a monitor and keyboard"
But what more specifically do you mean?

Specific example, my SlimServer box is untouched for months
at a time. Here is an uptime command from it as I type this.
22:59:39 up 160 days, 11:04, 2 users, load average: 0.01, 0.05, 0.07

I do everything I want to do to it remotely, usually from one or two
floors away. The machine has a KVM switch (keyboard, video, & mouse)
that I used to set it up many months ago.

Two way KVMs are cheap and available at your local CompUsa, Fry's, etc.
So if you call that headless, it is easy and cheap. If you are
really going low cost, you can just plug in the monitor/k/m from
your main PC during setup, saving even the $50.

Using this approach, you don't have to do anything magic at all.
Stick a prefered ISO disk, boot, install and be happy.


> Herger's SlimCD to manage a RAID system as well as run the slimserver;
> that would be an obvious win wrt what OS to install.

I agree with Jack's posting that RAID isn't necessarily what you want.
Backup is essential, but RAID isn't clearly the solution.


> Now onto the hardware side of things. The linux box should definitely
> have room for four hard drives, since RAID 5 hits a sweet spot at that
> number.

Any mid-tower will hold four or more drives, which you may want
even once you decide RAID is not the solution. Just remember
two key things:
1) disks run hot and hot disks are more failure prone,
2) disks require power, and low end power supplies are not
what you want for reliability.

> I don't know if the box should have an additional drive to host the OS
> and so forth.

I recommend it, as you can just nuke the OS disk and install another
one, mount your music disks and you are done. Just this is an
operational engineering decision.


> RAM is good and cheap and a gigabyte is a nice round number; that's
> another $100. Motherboard and cpu together will probably go for about
> $200;

This will get a far faster machine than a Slimserver requires.



> The noise reduction require a refined touch, as one has to travel
> inside the computer case looking for specific noise sources and then
> invest time, money and elbow grease to diminish or even eliminate them.

Only if you put the Slimserver in the listening room.
As I have posted elsewhere, the standard fans are cheap and noisy.
Spending an extra $5 per fan makes a huge difference.
My AMD 3400+ X64 system is very quiet, not silent, but
much less noisy than the Dell P3-933 mid-tower that
is sitting next to it in my office.

> 1. Fan quality makes a big difference; apparently one can invest for
> ~$10 rather than ~$3 fans and lose a lot of noise. But I've seen ads
> for fans that cost $30+ so I'm still working on this clue.

CPU fans that cost about $20 are fine, case fans that cost $15 and have
thermal speed controls are about right. Big fans are more expensive but
lots quieter at the same CFM than small ones.


> 3. If the linux box can be made to run without a VGA card, then that's
> an extra fan that doesn't have to run

You don't need a high end video card for a SlimServer. A nice
fanless, $25 video card is more than enough. It will be way
higher than VGA, probably at least 1600x1200x24



> So adding up the tab, we get motherboard, cpu, RAM, and case for about
> $400.

As I said, I got a fast CPU and 80GB of disks from my local
whitebox computer store for $400, and he installed all the
hardware and guarenteed it. It also included a DVD burner.

> Presumably this setup works better than, say, the ReadyNAS
> (albeit in a larger package), but one still has to rtfm and assemble
> (or hire someone else to do so, probably for another $100 or so).

Anyone can install Mandriva or Ubantu or Mike's SlimServerCD.
No need to pay anyone for that. But if you live in the Washington DC
area, you can pay me the $100 and I'll do it.


> The hard drives then pop in at one's leisure. The current sweet spot
> seems to be 400gb (after taking the fixed cost of the box into
> account), but the 300gb are pretty much in the same ballpark

You need to check speeds, faster drives are much louder.
I think paying more than $100 per drive is out of the sweet spot,
which is more like 250 GB each. But that is a large library.
My "724 albums with 10622 songs by 1193 artists" fits easily
in 250GB using FLAC.

> To sum up, I'd rather go DIY with the headless linux box for
> performance and compatibility/long-term reasons

And you can do it for a lot less than your $1300 budget number.

If you took my system and added water cooling, you could
still be in the $700 range, more if you water cool each drive.

I suggest you read the archives about backups. Backing up a terabyte
is not trivial and takes a long time. Just copying all the files
in one 250GB disk to another, internal to the computer, takes hours.

On which OS, the answer is pretty clear to me, you want
one of the free, well supported Linux distros. Not to
diss the *BSD folks, there are known good things about them,
but I think the Linux distros have a more approachable interface
for non-hardcore folks. Right now, Knoppix, Ubantu and Mandriva
are hot. I'm not a fan of RedHat/Fedora, but YMMV

I think you are on the right track, I just am not sure:
1) that really noiseless is cost justified
2) what you mean by headless
3) if RAID will really do what you want.


--
Pat
http://www.pfarrell.com/music/slimserver/slimsoftware.html

JJZolx
2005-11-26, 22:03
The reason that I haven't yet plunked down the cash for an Infrant box is that I'm not sure whether I'll be happy with it for reasons having to do with cpu cycles and noise. Since these boxes are not available for review at your local Fry's, you basically have to dive in and buy one before you personally experience it.

I have an Infrant 600 with four 320 GB WD drives. The thing is amazingly quiet. I didn't buy it to run SlimServer, but tried it out when I first got the NAS. It was a dog, but I know others are happy running SlimServer on the Infrant, so your mileage may vary. The biggest drawback is that the OS is not open to the user, so you you must run the SlimServer installation package that Infrant supplies. This means no nightlies, and few plugins.


Regarding the cpu cycles, I am concerned about how slimserver will run with a ReadyNAS (seeing how it eats cpu cycles in my venerable powerbook even with its 1 GB of RAM, it is something to think about); for example, waiting for several minutes or even hours before new additions to the music library show up ready to play. I have read the occasional complaint about this from ReadyNAS users so it's certainly a source of uncertainty.
Music doesn't just "show up" in your SlimServer library. If you're talking about running a scan, then yes, it may take hours. But you can always use Browse Music Folder to immediataly have new music added to the SlimServer database.


Regarding noise issues, if the NAS device is noisy then it defeats the audiophile raison d'etre of the squeezebox in an obvious way. The alternative is to place the NAS device in a separate room or somesuch, and rig up the network (e.g. good cable layout, satisfactory wireless performance) so that it's still accessible. Doable, but in the end a quiet device would certainly be preferable and give us more flexibility.
Only if the server is going to reside in the listening room is noise a huge issue. If the server is in another room, noise may not be a consideration at all. There's no reason you can't put the SlimServer in a basement, garage or office.


Now onto the hardware side of things. The linux box should definitely have room for four hard drives, since RAID 5 hits a sweet spot at that number. One wants to run RAID 5 because it provides a complete backup of the content of any one drive in the array, and because it does so without consuming so much hard drive space in the process. There are other RAID options and in some situations they are the way to go but for "us" it looks like RAID 5 is the clear choice and four drives is a good ergoecotechnomic minimax fit.
RAID 5 is _not_ a clear choice. More importantly, it does _NOT_ give you a backup of your data.

For moderately sized collections, I wouldn't recommend running RAID. It's nice if music is the be all and end all to your life. If a drive dies you keep on listening to music. That is the biggest advantage of RAID - to be able to continue running in the event of a hard disk failure. For moderately sized collections, a large drive can easily contain the entire collection. A second, equally sized (preferably external) drive can be used for backups. If drive A dies, replace it with drive B. Multiple drives can be used if the space is needed.


I don't know if the box should have an additional drive to host the OS and so forth. The advantage of that that I see is that one can then take out the drives and put them in some future container should the need arise, without violating the integrity of the past and future host boxen. Perhaps there is some performance advantage as well to having the system on one drive and the data on another (although I doubt that makes an important difference in our case). Since the DIY cases tend to have plenty of room it is certainly feasible to spend an extra $50 and get a 100GB+ hard drive on which to install linux and whatever other OS tickles your fancy. An extra hard drive not only raises the cost, however, it also raises the noise level (more on that below).
I find the OS drive plus data drive(s) approach attractive. Like you say, then you can take the data disks and move them to another system. If you're running software RAID, though, this may not be easy.


RAM is good and cheap and a gigabyte is a nice round number; that's another $100. Motherboard and cpu together will probably go for about $200; I had a look-around at the local Fry's and saw plenty of motherboards for about $100 that have integrated gigabit ethernet (which is helpful), but I haven't considered what it takes to run RAID proper.
If you really want RAID5, then there are some chipsets coming out now that offer onboard RAID5 and four SATA drive headers. I imagine that these aren't really hardware RAID per-se, but offload much of the processing onto the CPU. Or just use software RAID, or plan on a couple hundred dollars more for a decent RAID controller card.


3. If the linux box can be made to run without a VGA card, then that's an extra fan that doesn't have to run (not to mention a savings of a few bucks). Motherboards seem to have integrated vga now but I don't know if that means running an extra fan. Running headless is better all around wrt the "black box" approach to feeding music to the squeezebox, anyway.
Lot's of onboard video is available now. Even without it, you can throw some inexpensive (and fanless) graphics card into the machine.

If a silent machine is your ultimate goal, then you should also consider the chipset's north bridge, many of which require a fan. Many do not, however.

Here's a motherboard that I've been considering for a Linux SlimServer. It's targeted for HTPC computers, so has a lot of onboard systems. It's not a gamer's board, so the video isn't state of the art and there aren't many overclocking options in the BIOS.

http://www.newegg.com/Product/Product.asp?Item=N82E16813131570

- under $100
- socket 939 AMD
- onboard Gbit Ethernet
- onboard video
- onboard audio
- DDR 400 w/ support for ECC
- 4 SATA heaers, with RAID 0/1/0+1/5

The onboard audio might be used if you also were ripping CDs on the system and wanted to give the occasional listen to a rip. Or just disable it.


The hard drives then pop in at one's leisure. The current sweet spot seems to be 400gb
Nah, the sweet spot currently is still at 320/300GB or perhaps 250GB. If you don't run a large RAID 5 array, though, it may be worth the investment in a larger drive just so you don't have to replace it down the line. I think I'd look at the current size of my collection, then at how quickly I expected that collection to grow.

ceejay
2005-11-27, 02:16
An interesting discussion, the only drawback of which is that it covers so many points that this thread could very quickly fragment! Never mind, its a very good question, so here's my bit...

At the risk of repeating old RAID / non-RAID discussions, I'm not at all convinced that RAID is a good investment in this type of environment. In my day job I worry about availability of mission-critical enterprise computing systems, and one of the topics that keeps coming up is the inappropriate focus on technology to solve availability problems.

The key question for a slimserver is - what is most likely to cause data loss? (We shouldn't really be bothered about 24x7 uptime, unless we are using slimserver in say a hotel environment). I would assert that by far the most likely cause of data loss is human error - someone accidentally removes / overwrites / otherwise screws up the system.

The next most likely cause of loss will be software error. A typical slimserver is loaded up to the line with dozens of bits of software, designed by a host of different people, probably never been used in your unique combination, most of it without any formal support. Thr chances of, say, your tagging program crapping over your precious tag data are quite high.

And the least likely (though it will happen eventually) is hardware failure, and this is the only one that RAID will protect against.

If you want to guard against the first two, you have to keep some kind of separate backup of the data. And if you do that, you don't need RAID to protect against hardware error.

Personally I have a USB external drive which is NOT generally turned on or connected (to reduce the likelihood of getting corrupted) which I sync (currently using Windows synctoy) occasionally (when I think I've changed my library to the point that I'd be worried about loss).

YMMV !

Ceejay

docbee
2005-11-27, 02:39
I have a setup where "give me maximum capacity for the money" is the key objective. Therefore, I didn't go with a NAS because they are going to be quite expensive when you realize 2 + 1 TB as I did. However, here are my experiences with these servers, one having 14x160 GB harddisks in linux software raid 5 and the other one having 6x250 GB harddisks.

1. Noise. They are noisy but if you have a lan/wlan in your house most times you can find a room where 24/7 noise doesn't bother (cellar, untility room, etc)

2. Hard disks: I tried maxtor, wd and samsung. Samsung are the coolest and most silent from my experience. I can very much recommend the spinpoint P80 and P120 series. When you have that much drives you will notice that some of them will fail over time. After 2 years of non-stop service I usually sell the whole bunch on ebay and get a fresh new generation. As they have 3 years of warranty you don't take a great risk.

3. Board: My server boards are good old intel 815 with p3 1GHz or less. They work very stable and don't generate much heat. performance is very well for slimserver. even real time audio conversion tasks (alienbbc for example) work well.

4. Graphics: some 815 boards have graphics on board, if not just get an older nvidia card (TNT, TNT2, MX200, etc) without active cooling. They are cheap and work well with linux.

These servers are a bit outdated but they are very cost effective and generate much less heat than the actual setups. I am very sure that the TCO for 3 TB in total is much lesser than with any NAS. Having the full flexibility of linux and some processing power and as much ram as you like, these babies can do for another couple of years.

I strongly recommend not to look for a quiet server solution that fits into the living room. In order to get rid of the heat will certainly not work very well (at least if you are in the TB space). Find a place for them to run behind a door in a room where you can afford some airflow noise.

just my $0.02

trebejo
2005-11-27, 02:50
I've been withholding a reply to avoid thread fragmentation...

Very informative comments to y'all, thanks. Nice backup scripts. LA's too far away from DC, Pat, otherwise I'd take you up on the offer.

btw I agree about the *BSD v. linux approachable interface issues, but it should be mentioned at least in passing that I'm currently running a pretty approachable BSDish interface made by a Cupertino company.

Good to hear that the Readynas is AMAZINGLY quiet. Too bad it dogs it with slimserver. I suppose slimserver could run on some other machine on the network and let the Readynas handle the storage only, but not actually play the tunes. This, for people that delete a thread when it heads down the linux way.

wrt what I meant by "headless", I meant at the very least that it would run unattended without a monitor or keyboard. If it can reboot unattended and so forth that's all the better, since then I can get ambitious and actually park one of these things at my parents' and spread the joy. In either case, I'd run the slimserver either from another computer via a web browser or (and this feels like a stretch) solely with the remote control.

Sure, it's nice to just put the hardware away from the squeezebox with good wiring or wireless, but I'm not counting on having a garage or a basement available. In fact I'm probably looking at a small apartment in a few months. But it's good to heap on all the arguments in favor of putting the server far away from the audio, since in the end it's an imperative that can be addressed. There's always a closet and a long ethernet cable...

I'm surprised at how little you guys care for the RAID 5 idea. My problem is that at the moment I've got a couple of those external seagates (nice quiet gadgets and they look nice btw) and between the two of 'em they combine 700gb of room (ok 640gb actually) and they have about 30gb free combined. And that's after I backed up a little onto some DVDs. Perhaps it's the size of the collection that's the problem (I'm tagging like a madman so at the moment I only show the usual 900 album and 10000 songs but I think that by the time it's said and done I'll be around the 3000 album line). Rock 'n roll won't take up so much space, but classical is a toughie because you can always get another version of symphony X...

Anyway, back to the RAID issue. Doesn't the RAID do something about the occasional hard drive death? Like, if one dies, I can replace it and not lose any data? Yeah, sure, the big fire hits and all bets are off, but otherwise, isn't the RAID 5 setup quite a bit more robust than nothing at all? You guys make it sound like such a loser.

I suppose that in RAID, the hard drives never get a break, whereas in "regular" the one that is currently playing a song runs while the others idle? Ugh.

Ok. Use the RAID on a daily basis, and leave a bunch of these seagates in a closet in some other county as the data icebox. That means the same byte is written in three different places(!). Or don't run RAID at all, backup each drive in the server to the offshore seagate once in a while but only for the files that have changed; or some more sophisticated variation on that theme. btw Pat I think I was getting about 30GB/hour (so it'd take over 30 hours to do a terabyte--yipee) with the external seagates daisychained over firewire but my memory may be off; I'd run a test right now but when these guys get low on space they run a LOT slower so it'd be a bad reference. The box says they can do 400 Mbps but I never got close to that.

Ok, RAID or not, I need a box, it needs to run linux, it needs four hard drives inside it, and I need an offshore closet to sleep soundly at night. The rough outline is becoming focused.



Nah, the sweet spot currently is still at 320/300GB or perhaps 250GB. If you don't run a large RAID 5 array, though, it may be worth the investment in a larger drive just so you don't have to replace it down the line. I think I'd look at the current size of my collection, then at how quickly I expected that collection to grow.

It depends on the fixed cost of the supporting hardware. With Pat's setup, that cost goes down dramatically and I'll bet the 320gb is sweetest; with the ReadyNAS, it's a push (I'm using the prices I can get via pricewatch.com). As a stand-alone, yes, the 250gb is cheaper in $/gb but your box maxes out at a lower storage total and then when you want more than a terabyte you have to build another one. I put some numbers in the spreadsheet and they told me it was so. :)

http://forums.slimdevices.com/attachment.php?attachmentid=507&stc=1&d=1133083631

docbee
2005-11-27, 02:52
@ceejay: From my experience raid 5 (better raid 6) is vital when you have a certain number of drives. During the last 4 years I had about 2 drive failures a year on a total amount of about 15 drives running in total. This seems to be true regardless what kind of drive generation or manufacturer you have. I just can speak about PATA, no experience with SCSI which might be better but is out of my budget.

This failure rate was a surprise to me as I never had a drive failure on my desktop computers during the decade. For me the explanation is, that 24/7 has an impact and the presence of so many drives also raises the chance to see a drive failure.

If these drives are not on raid 5 I would have lost my media archive each year. This may be different on a baby NAS with 4 drives that only runs a few hours a day, but in larger setups raid 5 is a must, from my point of view.

JJZolx
2005-11-27, 04:15
I'm surprised at how little you guys care for the RAID 5 idea. My problem is that at the moment I've got a couple of those external seagates (nice quiet gadgets and they look nice btw) and between the two of 'em they combine 700gb of room (ok 640gb actually) and they have about 30gb free combined. And that's after I backed up a little onto some DVDs. Perhaps it's the size of the collection that's the problem (I'm tagging like a madman so at the moment I only show the usual 900 album and 10000 songs but I think that by the time it's said and done I'll be around the 3000 album line). Rock 'n roll won't take up so much space, but classical is a toughie because you can always get another version of symphony X...
Ok, for a library that large, RAID 5 makes a lot more sense.

The overall cost to run RAID 5 isn't that much greater than for non-RAID. It's essentially the cost of one extra drive. The more disks you use, the smaller this incremental cost.

It gives you the ability to have a single, very large volume, rather than dealing with many smaller single-disk volumes. This can make library management a lot easier.

Like docbee says, if you have a lot of hard drives, then you _will_ eventually lose some. You're mathematically multiplying the odds that it will happen. Running RAID makes this event less of an inconvenience.You're still going to need a backup strategy. I'm not sure what I'd do to backup a terrabyte or more of data. If you want to backup to disk, then assuming your array is 4 x 320GB disks, you'll need another 3 x 320GB of disk space for backup purposes. Thats a good $500 just for backup space, but that's mostly a factor of having such a large library. I can imagine the number of hours that will ultimately go into ripping and meticulously tagging 3000 albums, and $500 would be cheap insurance to keep from having to do it again.


Anyway, back to the RAID issue. Doesn't the RAID do something about the occasional hard drive death? Like, if one dies, I can replace it and not lose any data? Yeah, sure, the big fire hits and all bets are off, but otherwise, isn't the RAID 5 setup quite a bit more robust than nothing at all? You guys make it sound like such a loser.
Sure, there's no question that it's more robust, but the question is whether you need this kind of robustness for this type of application. For a home music server it's mostly just one of convenience, in that you can keep running after a single disk failure, and you don't have to spend time restoring from your backups (that you maintain religiously :-).

bernt
2005-11-27, 04:32
http://www.mashie.org/casemods/udat1.html

pfarrell
2005-11-27, 08:28
On Sun, 2005-11-27 at 01:50 -0800, trebejo wrote:
> I've been withholding a reply to avoid thread fragmentation...

Dreamer.

> wrt what I meant by "headless", I meant at the very least that it would
> run unattended without a monitor or keyboard. If it can reboot
> unattended and so forth that's all the better, since then I can get
> ambitious and actually park one of these things at my parents' and
> spread the joy. In either case, I'd run the slimserver either from
> another computer via a web browser or (and this feels like a stretch)
> solely with the remote control.

That is a easy definition to execute. Of course, as I posted in my
earlier msg, my slimserver box hardly ever is rebooted. 160
days is more than 5 months. Remote control is very much possible,
even my main server, which runs five domains, a serious webstore,
assorted mail, DNS and other services. You might want to get
a good X-terminal package for your real computer, Macs have
one built in.


> There's always a closet and a long ethernet cable...

If the apartnemt is that small, the cable won't be that long.


> I'm surprised at how little you guys care for the RAID 5 idea.

Its a question of engineering needs and approaches.
RAID 5 solves specific problems, but I don't see it
solving any problems that I have.


> Anyway, back to the RAID issue. Doesn't the RAID do something about the
> occasional hard drive death? Like, if one dies, I can replace it and not
> lose any data?

Sure, that is what it does best. But, and this is a big but,
do you know that one disk is gone? If you don't, and it
keeps working, there is a huge probability that you won't
do anything until the second drive dies. Then you are, ummmh,
this is a family list, in deep kimchi.


> I suppose that in RAID, the hard drives never get a break, whereas in
> "regular" the one that is currently playing a song runs while the
> others idle? Ugh.

Not sure what you are getting at here. The drives spin all the time,
the heads usually don't move unless they need to.

> . btw Pat I think I was getting
> about 30GB/hour (so it'd take over 30 hours to do a terabyte--yipee)

And another 30 hours to recover it. Just want people to know
that there are serious times involved.

Just a quick note in general on distros: I really like
two things about Mandriva. First, the GUI for system admin
is very nice, you can do almost everything from it
and not have shell configuration voodoo. Second,
it uses urpmi for updates, which resolves nearly all
RPM inter-dependencies. I used to hate RPMs with RedHat.

--
Pat
http://www.pfarrell.com/music/slimserver/slimsoftware.html

stinkingpig
2005-11-27, 08:38
....
> I'm surprised at how little you guys care for the RAID 5 idea. My
....
> Anyway, back to the RAID issue. Doesn't the RAID do something about the
> occasional hard drive death? Like, if one dies, I can replace it and not
> lose any data? Yeah, sure, the big fire hits and all bets are off, but
> otherwise, isn't the RAID 5 setup quite a bit more robust than nothing
> at all? You guys make it sound like such a loser.
>
> I suppose that in RAID, the hard drives never get a break, whereas in
> "regular" the one that is currently playing a song runs while the
> others idle? Ugh.
....

24x7 use is not that big a factor in my opinion, the heat of 4 drives in a
case designed for 1 hdd and some removable media drives is more likely
what kills the drives.

Anyway, I suppose one's disrespect of RAID is directly tied to the amount
of suffering one has incurred from rebuilding arrays and restoring from
backup after a failed rebuild :) RAID is inherently unreliable IMHO until
you reach 1:1 redundancy levels, and even then you're adding a layer of
cost and complexity over the simpler "buy two drives and copy the content"
method. Complexity == source of failures.

More about pain and suffering:
1) rebuilding a RAID 5 array after losing a drive takes a very long time.
Like days and days. I haven't tried it with a SOHO RAID card and some el
cheapo IDE drives, but top-of-the-line HPAQ and Dell cards with fast SCSI
drives sure do suck.
2) Lose two from a RAID 5 array, you've lost it all. Since people usually
build arrays by buying the same drives from the same manufacturer at the
same time, chances of losing two at once are remarkably high.
3) What about subtle drive errors which produce drive corruption? I once
lost a RAID 1 mirror because the failing drive caused file system
corruption on its way out. By the time that the drive actually failed
enough for LVM to notice and take it out of the array, the "RAID
protected" filesystem was trash. In "buy two drives and copy the content",
the copy fails when one of the drives doesn't work perfectly. If you're
using hard link backups, you don't trash the copy, so you still have a
backup.

--
Jack At Monkeynoodle.Org: It's A Scientific Venture...
"Believe what you're told; there'd be chaos if everyone thought for
themselves." -- Top Dog hotdog stand, Berkeley, CA

dem
2005-11-27, 14:21
More about pain and suffering:
1) rebuilding a RAID 5 array after losing a drive takes a very long time.
Like days and days. I haven't tried it with a SOHO RAID card and some el
cheapo IDE drives, but top-of-the-line HPAQ and Dell cards with fast SCSI
drives sure do suck.

Yikes, that does suck.

A couple years ago I built my own NAS box using a Mini-ITX board, a 3ware RAID card, and 4 Seagate 160 GB PATA drives (which seemed huge at the time) running Fedora Linux. One of the drives failed after a few months and I was able to replace it without any downtime, but the rebuild only took an hour.

I've been very pleased with how the RAID card has performed, but I agree with others here that one shouldn't depend on RAID for a backup. I use the rsync command to back up the music files on the NAS box to an external USB drive, then I keep the USB drive in a fire safe. There's no way I want to go through all that ripping and tagging again.

Stephen Ryan
2005-11-27, 15:27
On Sun, 2005-11-27 at 01:50 -0800, trebejo wrote:
> I'm surprised at how little you guys care for the RAID 5 idea.
....
> Anyway, back to the RAID issue. Doesn't the RAID do something about the
> occasional hard drive death? Like, if one dies, I can replace it and not
> lose any data? Yeah, sure, the big fire hits and all bets are off, but
> otherwise, isn't the RAID 5 setup quite a bit more robust than nothing
> at all? You guys make it sound like such a loser.

Yes, it is more robust than nothing at all; the concern is that there
are significant, broad failure classes that RAID does nothing at all to
address, and that you might a) get sloppy because you think RAID
protects you, or b) run out of money. Provided that you don't spend
money setting up RAID 5 that you should be spending on your off-site
solution, and that you don't put off the off-site solution because you
get lulled into a false sense of security because of the RAID setup,
then yes, RAID is a grand idea.

All of my machines at work have the data stored on a RAID-1 array, and
when I get to replace the hardware in a few weeks, the new machines are
going to use RAID-1 for everything. Yes, we're paranoid, but it does
make recovering from a hard drive failure *much* easier and faster, and
it has saved us a significant amount of time and effort in the past, so
my boss is convinced.

You should be aware that there is a performance penalty for using RAID5;
every time you write a sector to disk, you must read the same sector
from each of the other drives in your array and recompute the checksums.
For a music library that doesn't change much (i.e., many more reads than
writes), you won't pay too much of a penalty for using RAID5. If you
intend to use the headless server for other applications as well, it may
be worth investigating either RAID 1 or RAID 0+1 instead (RAID 0 is, I
think, worse than useless, but it looks like you already know that).

I'd also install smartmontools as a way of getting at least some
information out of the server about the health of the hard drives, and
something like logcheck to automatically email the results to you so
that you don't have to remember to check.

stinkingpig
2005-11-28, 08:34
>
> stinkingpig Wrote:
>>
>> More about pain and suffering:
>> 1) rebuilding a RAID 5 array after losing a drive takes a very long
>> time.
>> Like days and days. I haven't tried it with a SOHO RAID card and some
>> el
>> cheapo IDE drives, but top-of-the-line HPAQ and Dell cards with fast
>> SCSI
>> drives sure do suck.
>>
> Yikes, that does suck.
>
> A couple years ago I built my own NAS box using a Mini-ITX board, a
> 3ware RAID card, and 4 Seagate 160 GB PATA drives (which seemed huge at
> the time) running Fedora Linux. One of the drives failed after a few
> months and I was able to replace it without any downtime, but the
> rebuild only took an hour.
>

This system had nearly a terabyte of small files (maildir email, mmmmm, so
chewy), and it had to have been 3 years ago, so in fairness maybe things
have gotten better in the RAID world. Still, I think we've all made it
clear that it ain't the same as a backup copy :)

--
Jack At Monkeynoodle.Org: It's A Scientific Venture...
"Believe what you're told; there'd be chaos if everyone thought for
themselves." -- Top Dog hotdog stand, Berkeley, CA

jimdibb
2005-11-28, 09:01
To make a small correction to what's below, you do not have to read the same
sector from every other drive to write to one drive. You only have to read
the sector you are writing to and the corresponding parity sector. So,
there's still a penalty, but not as large as what's suggested below.

On 11/27/05, Stephen Ryan <steve (AT) deepthought (DOT) dartmouth.edu> wrote:
>
> On Sun, 2005-11-27 at 01:50 -0800, trebejo wrote:
> > I'm surprised at how little you guys care for the RAID 5 idea.
> ...
>
> You should be aware that there is a performance penalty for using RAID5;
> every time you write a sector to disk, you must read the same sector
> from each of the other drives in your array and recompute the checksums.
> For a music library that doesn't change much (i.e., many more reads than
> writes), you won't pay too much of a penalty for using RAID5. If you
> intend to use the headless server for other applications as well, it may
> be worth investigating either RAID 1 or RAID 0+1 instead (RAID 0 is, I
> think, worse than useless, but it looks like you already know that).
>
>

docbee
2005-11-28, 11:49
24x7 use is not that big a factor in my opinion, the heat of 4 drives in a case designed for 1 hdd and some removable media drives is more likely what kills the drives.
sorry, not true from my experience. if a drive runs 24/7 for 2-3 years you really get a chance to see a failure. As posted befroe, this was quite a surprise to me as well...


1) rebuilding a RAID 5 array after losing a drive takes a very long time.Like days and days. I haven't tried it with a SOHO RAID card and some el cheapo IDE drives, but top-of-the-line HPAQ and Dell cards with fast SCSI drives sure do suck.
true , but doesn't care. The RAID 5 is fully functional during degraded mode (lost one drive) and during resyncing a new one. THis is really a great feature that minimizes downtime.


2) Lose two from a RAID 5 array, you've lost it all. Since people usually build arrays by buying the same drives from the same manufacturer at the same time, chances of losing two at once are remarkably high.
True, but if you use RAID 6 the system can cope with two faulty drives. RAID 6 comes for free as software raid with kernel 2.6 and it works great.


3) What about subtle drive errors which produce drive corruption? I once lost a RAID 1 mirror because the failing drive caused file system corruption on its way out. By the time that the drive actually failed enough for LVM to notice and take it out of the array, the "RAID protected" filesystem was trash. In "buy two drives and copy the content", the copy fails when one of the drives doesn't work perfectly. If you're using hard link backups, you don't trash the copy, so you still have a backup.
You are right, if the drive does bad things without raising error conditions on the IDE channel, than you are in hell. But that is a risk you do always take. For example, if you do copy the drive to a backup drive on a device-level, than you also get the inconsistent filesystem on the backup and your are lost as well. If you do it on a filesystem level than you already have replaced a part of your backup when you get aware of the problem. So this problem is not raid specific. Up to now I did not experience this type of error with my raids. But may be I just had luck :-)

The advice to copy the whole stuff to a second drive is a bit hard to realize when you have 3 TB to move around. But may be that is my specific problem as well :-))

jackaninny
2005-11-28, 12:04
panaflo fans are the best - period - end of discussion. each size (80mm, 92mm etc etc) has 3 speeds you can get - the low speed units provide MORE than enough airflow for most applications. panaflo is more expensive but they are VERY quite and last forever.

here endith the lesson.

Stephen Ryan
2005-11-28, 12:18
Yes, you're absolutely right. Sorry, brain spazz on my end.

On Mon, 2005-11-28 at 11:01 -0500, Jim Dibb wrote:
> To make a small correction to what's below, you do not have to read
> the same sector from every other drive to write to one drive. You
> only have to read the sector you are writing to and the corresponding
> parity sector. So, there's still a penalty, but not as large as
> what's suggested below.
>
> On 11/27/05, Stephen Ryan <steve (AT) deepthought (DOT) dartmouth.edu > wrote:
> On Sun, 2005-11-27 at 01:50 -0800, trebejo wrote:
> > I'm surprised at how little you guys care for the RAID 5
> idea.
> ...
>
> You should be aware that there is a performance penalty for
> using RAID5;
> every time you write a sector to disk, you must read the same
> sector
> from each of the other drives in your array and recompute the
> checksums.
> For a music library that doesn't change much (i.e., many more
> reads than
> writes), you won't pay too much of a penalty for using RAID5.
> If you
> intend to use the headless server for other applications as
> well, it may
> be worth investigating either RAID 1 or RAID 0+1 instead (RAID
> 0 is, I
> think, worse than useless, but it looks like you already know
> that).
>
>
>

notanatheist
2005-12-05, 00:14
Yikes, that does suck.

A couple years ago I built my own NAS box using a Mini-ITX board, a 3ware RAID card, and 4 Seagate 160 GB PATA drives (which seemed huge at the time) running Fedora Linux. One of the drives failed after a few months and I was able to replace it without any downtime, but the rebuild only took an hour.


Dem, you're good. 3Ware is *the* way to go for a RAID setup on an existing PC. The support in linux is there. No freakin' softRAID. I use softRAID on my present file server but it's only RAID1. If I was going to upgrade it I'd go 3Ware and RAID5.

As for the Infrant boxes, why run SlimServer on them? Why not throw together a really cheap linux box with a GBit NIC, NFS mount the NAS, then share it!! That'd be smokin'. That's my thoughts and intentions.

trebejo
2005-12-12, 02:52
First of all, I'd like to thank y'all for the very informative thread. I hope other people find it useful.

On my end, after mulling things over and feeling quite lazy about the linux rtfm requirement, I've decided to go with a pins-and-needles-and-mac setup. I'm going to use firewire drives attached to an apple box (a mini or an old g4 tower would do), and add drives as needed. I chose firewire over usb 2.0 because of performance considerations and because it seems that the firewire approach lends itself more readily to a large external array. There will be a long firewire cable with repeaters leading to that distant closet if I use "my box", or an ethernet cable running out to the router if I get a dedicated apple box to sit in the closet with the drives; the drives will hang off a firewire hub with 30" cables so as to shorten the cable distance between any two nodes as well as reducing clutter. Each new hard drive will be cloned and the clone (sans enclosure) will be kept in a separate building; I'll deal with the "partially-filled" drive in some way, but I'm not so worried about the fragility of its data compared to that of the rest.

Since the music collection consists of a monotonically increasing filesystem, this fairly brainless backup strategy should be good enough.

One definite shift in my thinking that happened thanks to this thread is that now I will not assume that RAID protects my data for the long-term as securely as I'd like.

I've tried to be picky with the enclosures for these firewires; I've ordered one of each of these to see which one I'll stick with for the long haul:

http://www.supergooddeal.com/product_p/pm-350u2-bk--xx.htm
http://www.censuspc.com/product.php?productid=3036&cat=72&page=1

The first one is smaller and less expensive, whereas the second looks like "good engineering" and I like its big fan. I'll try them both and take it from there. I'm going to need a case that is a kissing cousin of "hot swappable" for the cloning of the barebone drives; hopefully both of these can fulfill that function, so that the one that doesn't get to be the array model can still be the cloner. :)

Wrt other external choices, the pair of seagates that I've been using are quiet enough to be in the listening room and they have worked more or less flawlessly for me:

http://www.seagate.com/products/retail/external/usbfirewire

(one of them had the inexplicable dismount a few weeks ago and OS X wouldn't recognize it initially but a quick look at it with a disk utility repaired it without a problem). They look pretty, too. However they do heat up a little bit (I wouldn't stack them as the picture suggests, but use them vertically and keep a gap between them). I also began to think that in the long term, if the case fails, I may have a bit of difficulty extracting the hard drive from within... that's when I realized that if I go with a barebone hard drive and a separately-chosen enclosure, I double the points of failure and should one component fail, I still get to keep the other hardware component (assuming the failure is not as catastrophic as, say, a fire).

Aside from the brainless simplicity of this setup, I'm hoping that it'll perform better than the readynas as a slimserver platform, and of course at about $50/slot it is cheaper than the $600-700 for the four-drive readynas.

btw if I'd done the linux box in the closet path, I would have picked this case

http://www.thermaltake.com/xaserCase/armor/va8000swa/va8000swa.htm

along with these sexy thangs,

http://www.thermaltake.com/xaserCase/icage/icage.htm

Of course this would have busted Pat's $400 budget. :)

Cheers,

Ariel

PoW
2005-12-15, 14:10
I saw mashie's uDat come up a few posts ago and had to mention this. Exactly a year ago now I built my pDat, a uDat clone.

Here is a work log: http://www.hardforum.com/showthread.php?t=845672

It works exceptionally well however I have had some terrible heat problems...3 drive failures. It is always the drive on one side of the enclosure and it is the result of the system being in an environment over 75deg or so. This excessive heat roasts the drive on the side and kills it.

Being raid 5 I have not lost any data and having the drives all under warrenty is a big plus. I am currently designing a cooling chamber of some sort to keep the ambient temp around 60 degrees or so.