PDA

View Full Version : Rescan via wondows takes forever



egd
2007-01-18, 19:09
I'm in the process of adding some hardware to my Linux box so have had to take it down for the task. As an interim measure I initiated a rescan/look for changes via a windows server box running 6.5.1/10833. The existing db was about 300-350 albums out of date but the rescan is still going some 90 minutes later. Under Linux the same operation takes a few minutes and a total rescan takes around 10. What gives?

Skunk
2007-01-18, 19:47
Mine are under five minutes on Win2k for 150 albums, so that's on par with the linux box I suppose.

I checked d_scan and did a wipe+rescan and got this:
21:28:06.7720 Setup::rescan - initiating scan of type: [wipecache]
which I assume means nothing-

but noticed this above it which appears to be from when I installed 6.5.1 ten minutes ago.:
21:18:38.1875 Warning: Migrating from 6.3.x used with MySQL!

JJZolx
2007-01-18, 21:08
I'm in the process of adding some hardware to my Linux box so have had to take it down for the task. As an interim measure I initiated a rescan/look for changes via a windows server box running 6.5.1/10833. The existing db was about 300-350 albums out of date but the rescan is still going some 90 minutes later. Under Linux the same operation takes a few minutes and a total rescan takes around 10. What gives?

If it never finishes, then it's not a matter of being slow - it would be something hanging or crashing the scanner.

How many files, where are they stored and what are the comparable hardware platforms? When I had my files located on an NAS, scanning from Windows took more than 35 minutes. Moving the files onto the Windows machine itself shortened that to under 12. I'm not sure if that's par for having the files located on a file server, or if there are Perl/Windows issues reading files across the network.

egd
2007-01-18, 21:22
How many files, where are they stored and what are the comparable hardware platforms? When I had my files located on an NAS, scanning from Windows took more than 35 minutes. Moving the files onto the Windows machine itself shortened that to under 12. I'm not sure if that's par for having the files located on a file server, or if there are Perl/Windows issues reading files across the network.

Its close to 3,000 albums and just over 40,000 songs located on a NAS connected via a gigabit switch. From Linux a full rescan takes under 10 minutes and a refresh takes only a few minutes. Both machines are recent P4's, 1GB ram, SATAII hdd's etc.

I think in my case it is going to be best to do a full rescan. Gotta love windows.

JJZolx
2007-01-18, 21:40
Its close to 3,000 albums and just over 40,000 songs located on a NAS connected via a gigabit switch. From Linux a full rescan takes under 10 minutes and a refresh takes only a few minutes. Both machines are recent P4's, 1GB ram, SATAII hdd's etc.

I think in my case it is going to be best to do a full rescan. Gotta love windows.

Same thing I experienced when I ran SlimServer on Windows with the files on an NAS and a gigabit network. Yours is a good comparison case, since you've scanned from both Linux and Windows. Something seems to be wrong with either the way Perl on Windows accesses the network, or else it's something SlimServer is doing on the Windows platform that makes it so slow.

http://bugs.slimdevices.com/show_bug.cgi?id=3896

It would also be nice to see Slim Devices targeting Windows better and doing more testing on that platform to keep issues like this from happening.

egd
2007-01-19, 22:43
It would also be nice to see Slim Devices targeting Windows better and doing more testing on that platform to keep issues like this from happening.

I've installed 7.0a1/11234 and rescanned and th outcome is the same - 2 hours later the scan is apparently still going.

egd
2007-01-20, 07:18
I've installed 7.0a1/11234 and rescanned and th outcome is the same - 2 hours later the scan is apparently still going.


The rescan finally finished and I've been able to listen for a few hours. Having now had the opportunity to compare slimserver under Linux and Windows I really have no option but to conclude that slimserver under windows is very sluggish by comparison - for rescans and running in general. It also appears to be buggier with a few freeze's /grinds to a halt this evening already. I'd better get that Ubuntu box back up and running tonight. Anyone who has not run slimserver under Linux will likely be very pleasantly surprised by the difference in performance and stability. Not sure whether the cause lies with slimserver or the os', though I suspect a bit of both.

shabbs
2007-01-20, 07:43
Its close to 3,000 albums and just over 40,000 songs located on a NAS connected via a gigabit switch. From Linux a full rescan takes under 10 minutes and a refresh takes only a few minutes. Both machines are recent P4's, 1GB ram, SATAII hdd's etc.

I think in my case it is going to be best to do a full rescan. Gotta love windows.
That is very interesting. A really good direct comparison. I think I may have to start looking at migrating to Linux.

Cheers.

egd
2007-01-20, 07:55
I think I may have to start looking at migrating to Linux.

If you're not familiar with Linux might I suggest that you go with Ubuntu - it is a pretty user friendly distribution and easy to configure for slimserver.

shabbs
2007-01-20, 08:12
If you're not familiar with Linux might I suggest that you go with Ubuntu - it is a pretty user friendly distribution and easy to configure for slimserver.
No worries... been down the Linux road many a time. RedHat, Suse, Debian, Mandrake, Slackware, Yellow Dog... but it's been a while. Will have to take a look at Ubuntu.

Cheers.

jonheal
2007-01-20, 12:56
The rescan finally finished and I've been able to listen for a few hours. Having now had the opportunity to compare slimserver under Linux and Windows I really have no option but to conclude that slimserver under windows is very sluggish by comparison - for rescans and running in general. It also appears to be buggier with a few freeze's /grinds to a halt this evening already. I'd better get that Ubuntu box back up and running tonight. Anyone who has not run slimserver under Linux will likely be very pleasantly surprised by the difference in performance and stability. Not sure whether the cause lies with slimserver or the os', though I suspect a bit of both.

Is your music stored on a machine OTHER than the one running Windows and Slimerver? Mine is and it takes a long time to scan. I chalk up the extra time to whatever is dealing with the drive mapping.

shabbs
2007-01-20, 13:19
Is your music stored on a machine OTHER than the one running Windows and Slimerver? Mine is and it takes a long time to scan. I chalk up the extra time to whatever is dealing with the drive mapping.
He's got his music on a NAS device connected via a gigabit switch. The key thing is that for him, on Linux, the same scan takes only 10 minutes. Which is an incredible difference from the hours it takes on Windows. Is it the way Windows handles the NAS device? Or the Windows network drivers? Or the Windows SlimServer code? Several things could come into play.

Cheers.

JJZolx
2007-01-20, 13:39
He's got his music on a NAS device connected via a gigabit switch. The key thing is that for him, on Linux, the same scan takes only 10 minutes. Which is an incredible difference from the hours it takes on Windows. Is it the way Windows handles the NAS device? Or the Windows network drivers? Or the Windows SlimServer code? Several things could come into play.

I doubt very much that it's Windows itself. When I had my files on the Infrant NAS it wasn't blazing by any means, but copying gigabytes at a time to and from the NAS was acceptably fast and the bottleneck was definitely the anemic Infrant device. Windows network file access has been around for a _long_ time and other programs demostrate perfectly fine, fast file transfer speeds.

It could be ActiveState's ActivePerl, but again, it's been around a while and if it were really _that_ slow, someone would probably have noticed by now.

My bet is that it's something in SlimServer when run on Windows.

mherger
2007-01-20, 14:22
> My bet is that it's something in SlimServer when run on Windows.

Or something with playlists created on Linux, therefore containing paths
that don't exist on Windows. SlimServer can run fast on Windows. My 1.4GHz
Laptop with Windows (and a slow disk) is scanning faster than my 1GHz
Via/C3 Linux server.

It's not "Windows vs. Linux" nor "Perl vs. a real language", but some odd
behaviour with the OP's particular configuration.

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR

shabbs
2007-01-20, 14:56
It's not "Windows vs. Linux" nor "Perl vs. a real language", but some odd behaviour with the OP's particular configuration.
Is the best way to diagnose that using the d_scan flag and debug?

Cheers.

JJZolx
2007-01-20, 15:08
It's not "Windows vs. Linux" nor "Perl vs. a real language", but some odd
behaviour with the OP's particular configuration.

Exactly. But somehow I don't think it's that odd. The crux of the setup is that the files are stored on a network file share. I've also experienced it, and at least one other guy as well. I don't have any playlists.

To the original poster: What kind of NAS is it?

It would be interesting to put together a test bed and see what the particulars are. For instance, if the file server is itself running Windows, is it just as slow? It would also be good to find out exactly where the slowdown is - is it when pulling files across the network to analyze headers and tags? Does SS normally need the entire file, or does it try to pull just the peices it needs?

egd
2007-01-21, 01:39
Is your music stored on a machine OTHER than the one running Windows and Slimerver? Mine is and it takes a long time to scan. I chalk up the extra time to whatever is dealing with the drive mapping.

Yes, slimserver is installed on a w2k3 server connected via a gigabit switch to the NAS holding the audio. Identical setup on Linux connected to the same NAS without the performance issues.

egd
2007-01-21, 01:41
Or something with playlists created on Linux, therefore containing paths
that don't exist on Windows.

There are no playlists on the NAS or elsewhere, so I'm afraid it is not that.

egd
2007-01-21, 01:47
To the original poster: What kind of NAS is it?

It would be interesting to put together a test bed and see what the particulars are. For instance, if the file server is itself running Windows, is it just as slow? It would also be good to find out exactly where the slowdown is - is it when pulling files across the network to analyze headers and tags? Does SS normally need the entire file, or does it try to pull just the peices it needs?

Infrant ReadyNAS. Doing a straight copy of music from the windows box to the NAS gets througput of around 12MB/s, which is far from their claimed benchmark, but acceptable for my purposes. I'm comfortably able to edit tags etc. residing on the NAS from the windows box.

The performance issue definitely does not lie with the NAS or connectivity to the NAS. Can't be CIFS either otherwise copy operations etc. would be dog slow too. I reckon it is a slimserver issue, albeit I find NFS inherently faster than CIFS hence partially accounting for the Linux box performing better.

I added more albums to the NAS today and started rescan/changes only on Windows about 2 hours ago. It is now at the 50% mark. Before it is finished I'll have my Ubuntu workstation up and running again, install latest dev build of slimserver and do a full rescan. I guarantee I'll be listening to music via the Linux instance before the windows instance reaches 60% on the rescan. I'll time the Linux instance rescan (full scan) and report the outcome here.

egd
2007-01-21, 12:37
I'll time the Linux instance rescan (full scan) and report the outcome here.

A full rescan from a fresh install on my Ubuntu box running the most recent 7.x took around 25 minutes to yield 3156 albums and over 43,000 tracks, including the artwork pass. The scan was done whilst the windows instance was 1) doing a refresh/look for changes and 2) playing tracks.

There is definitely a huge performance difference, even if slimserver is idle and all you're doing is browsing artists/albums. My windows instance hesitates, whereas the linux instance responds almost instantaneously.

egd
2007-01-26, 19:03
Edit: here's a screenshot of a recent rescan/changes only operation that shows just how efficient slimserver's scanner is under Linux.

snarlydwarf
2007-01-26, 19:11
oh man, that's pretty... forget the times, now you are making me lust after 7.0.

must resist mindless upgrade until 7.0 is a bit more stable.. must resist...

mherger
2007-01-26, 23:12
> must resist mindless upgrade until 7.0 is a bit more stable.. must
> resist...

Yeah... I wanted to tell ebd _not_ to show this for exactly that reason.
And I won't publish the fanciest MusicInfoSCR ever for a while for the
exact same reason :-)

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR