PDA

View Full Version : Library Scan Very Slow



hal9000
2008-03-05, 18:47
I just transferred all of my FLAC files (almost 4K albums, 1K artists and 55K songs) on to a 4TB Buffalo NAS and installed SlimCenter 7. I am running the server on a Win XP SP2 high end computer. The scan took a full 24 hours to complete.

I did a search on slow scans and saw that this was an issue with NAS storage. Has it been resolved or a work around found?

haunyack
2008-03-05, 18:52
Love the avatar.
Have you checked the scan log for indication of problems?

\\Documents and Settings\All Users\Application Data\SqueezeCenter\Logs\Scanner.log
.

pfarrell
2008-03-05, 18:59
hal9000 wrote:
> I did a search on slow scans and saw that this was an issue with NAS
> storage. Has it been resolved or a work around found?

The usual solution is not likely to be what you want to hear:
get a real computer with some power. Or, and this is what most folks do,
just go watch TV, talk to your kids, or something else to distract you.

Most NAS boxes are just braindead PCs with slow CPUs and tiny memory.
It will finish, and then it will probably be fast enough. If not, grab
an old PC....

--
Pat Farrell
http://www.pfarrell.com/

hal9000
2008-03-05, 20:05
Thanks guys. I'll check out my log and see if there are issues there. The PC I'm running the server on is a buffed out XPS gaming machine from last year so that's not the problem...its probably the NAS or a network issue.

JJZolx
2008-03-05, 20:34
Thanks guys. I'll check out my log and see if there are issues there. The PC I'm running the server on is a buffed out XPS gaming machine from last year so that's not the problem...its probably the NAS or a network issue.

You may have found my previous posts on this in your searches, but that's the same sort of thing I found running the server on a Windows XP machine with the library on an Infrant ReadyNAS across a gigabit netwwork. Linux systems appear to scan much faster when the files reside on a Linux based NAS, but I've never tested this myself.

Something is broken with Windows/Perl on Windows/SlimServer in this scenario, but where exactly the breakdown lies, nobody seems to know.

I think your choices boil down to:

- store the music library on local disks (and use the NAS for backups, perhaps)

- run Linux on the server

- use a Windows based NAS or file server

iPhone
2008-03-05, 22:09
Thanks guys. I'll check out my log and see if there are issues there. The PC I'm running the server on is a buffed out XPS gaming machine from last year so that's not the problem...its probably the NAS or a network issue.

Sounds like your PC is fine. The slow part is your NAS. That type of NAS was not made to deal with 4TB of FLAC files during scanning. Just think of how long it would have taken with SL 6.5.4. The hard part is over, the scan. It should do all right playing the FLAC music files. Just because a certain NAS has enough capacity, does not necessarily make it the correct NAS for Squeezebox file sharing no matter how powerful the PC running SqueezeCenter is. And it is a sure thing not to use it as the actual SC server.

I just finished installing SqueezeCenter on my DIY NAS. The rescan took almost 4 hours to do just under 12,000 CDs (still have 4000 to rip to FLAC) spanned over 6TB of hard drive space. The NAS has a 3.2 GHz Pentium 4 with 4 GB of RAM and an LSI Logic SATA 150-6 RAID Card so it can stand along as the SqueezeCenter server. My NAS has a huge amount of horsepower. The reasons are two fold. I have an abnormally huge CD library and I want my NAS to host SqueezeCenter on its own.

Large FLAC libraries have to have an extremely powerful NAS if they are going to also run as the Server (unless time is not important to you). Large libraries require a powerful NAS when used for storage only to handle maintenance/scanning while ending up having excess power when just playing the music files. What was commercially available (other then rack mounted huge things, read very expensive) just would not handle my collection so I went with the DIY NAS. Not including drives, I stayed under $600 if purchased new (spent less because I had stuff in the closet).

pfarrell
2008-03-05, 22:20
iPhone wrote:
> Sounds like your PC is fine. The slow part is your NAS. That type of
> NAS was not made to deal with 4TB of FLAC files during scanning.

Actually, its safe to say that no consumer NAS is going to have good
performance during a library scan. To pull the tags out of the music
files, you have to read the files. Sometimes you can just read the
start, sometimes the end, sometimes all of it, depending on file type,
tag type, etc.

Here is a simple experiment to give you an idea:
Copy a couple of albums worth of files from one directory on the disk to
another. Use about a gigabyte of files (say four FLAC encoded albums).

See how long it takes. Then multiply that by one thousand times the
number of terabytes you have. Its a really, really big number.

Way back when I was working at a streaming music company, and we scanned
40,000 CDs, we used to use swappable disk drives to move files, rather
than using a 100baseT Ethernet. It was much faster to shut down the two
computers, swap out the drive, carry it 10 feet, and power up the
computer than it was to copy a 40 GB disk. Disks are only slightly
faster now, and they are much, much bigger

Pat

--
Pat Farrell
http://www.pfarrell.com/

hal9000
2008-03-05, 22:22
I've been an SB freak for almost two years now and the first thing I always point out when trying to sell the system to a new user is the amazing community here. These posts are VERY helpful.

iPhone - Where were you when I bought my NAS? ;) In a few months I'm going to get/build a DIY NAS with a buffed out processor for my primary and use the Buffalo as a back up server. I didn't know you could get/build a NAS with that kind of processing muscle.

For now, I can live with the slow rescan times as I don't notice a serious delay in the file serving when I play my FLAC files.