PDA

View Full Version : Integrity/reliability of library scan process?



egd
2007-06-28, 23:25
I have just completed a full rescan of my audio library under windows and am getting a rather large discrepancy between slimserver's scan results and a file (*.flac) and folder count on my audio library.

The folders are arranged as follows:
AudioLIB/<alphabet range>/artist - album/"artist - album - track - title.flac"

where alphabet range comprises:0-E, K-O, P-R, S-T & U-Z.

According to Windows there are 65,803 FLAC files in 4,484 folders
According to slimserver there are 114,297 songs across 4,451 albums . All files are tagged and there are no cue or playlist files in the library. I know slimserver is incorrect. What gives?

Out of curiosity I'll initiate a scan of the same library from Linux and report the results when it's done.

[Edit]Linux result: Your music library contains 4449 albums with 65721 songs. A closer match, now to find the missing 35 albums and 82 tracks. Needle in a haystack...

Incredibly slimserver running on windows is STILL rescanning despite the library itself residing on the windows box. Who'd have thought a scan over a SAMBA connection would beat a local scan. Can anyone explain this?

egd
2007-06-29, 03:14
windows rescan completed reporting 4452 albums with 65795 songs.

How is it that two instances of slimserver scanning the same library derives different results?

egd
2007-07-02, 03:42
How is it that two instances of slimserver scanning the same library derives different results?

bump...anyone, developers, contributors ...?

mherger
2007-07-02, 03:49
>> How is it that two instances of slimserver scanning the same library
>> derives different results?
>
> bump...anyone, developers, contributors ...?

If one is launched by the server, run as a service, it has different
permissions than being rund by the user in its context. This can result in
different results if some of the files aren't accessible for the user, but
not the system's services.

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR

mherger
2007-07-02, 03:52
>> How is it that two instances of slimserver scanning the same library
>> derives different results?
>
> bump...anyone, developers, contributors ...?

Oh, (after reading your initial posting) - do you mean why the difference
between the Windows and the Linux result? Do you have playlists in the
folders pointing to some of the song files?

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR

egd
2007-07-02, 05:45
Oh, (after reading your initial posting) - do you mean why the difference
between the Windows and the Linux result? Do you have playlists in the
folders pointing to some of the song files?

Yes, why does a scan from Linux/slimserver across SAMBA produce a different result to Windows/slimserver scanning the same library residing on a RAID array connected directly to the Windows box? Re playlists - none whatsoever, not playlists, not cue files, nothing but FLAC files and folder.jpg in each folder.

mherger
2007-07-02, 06:08
> Yes, why does a scan from Linux/slimserver across SAMBA produce a
> different result to Windows/slimserver scanning the same library
> residing on a RAID array connected directly to the Windows box?

Permission issues? 3rd party management tools like iTunes, MusicIP?

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR

Pale Blue Ego
2007-07-02, 09:05
It seems like every piece of software I point at the library comes up with slightly different numbers.

I quit worrying about it a long time ago.

hornseaey
2007-07-02, 10:08
I discovered that some files were not being indexed at all and that some had wrong information. I had tagged the latter but they were indexed with completely different details. I examined the files and discovered that the "wrong" information was contained in an APE tag at the end of the file. Haven't discovered why some are missing completely. Could this explain the difference between the stats and actual number of files?

egd
2007-07-03, 08:00
Permission issues? 3rd party management tools like iTunes, MusicIP?

No 3rd party management tools - all MIP data is archived to tags. The RAID share is mounted R/W in Linux using the Windows box's Administrator credentials.


sudo mount -t smbfs //p5wdg2wspro/AudioLIB /media/p5wdg2wspro/ -o username=uname,password=pw,dmask=777,fmask=777


Could this explain the difference between the stats and actual number of files?

Possibly, though I don't believe I have any nonconforming tags (all created with mp3tag) and I would have thought slimserver would return the same result, irrespective of which OS it is running on - the underlying code is the same.

JJZolx
2007-07-03, 08:25
Possibly, though I don't believe I have any nonconforming tags (all created with mp3tag) and I would have thought slimserver would return the same result, irrespective of which OS it is running on - the underlying code is the same.
The bigger difference is probably that you're running one scan with local files and another across a network share. What happened to that 114,000 songs number that you reported initially. How did it end up going down?

What version of SlimServer are you running on the two machines?

egd
2007-07-03, 17:45
The bigger difference is probably that you're running one scan with local files and another across a network share. What happened to that 114,000 songs number that you reported initially. How did it end up going down?

What version of SlimServer are you running on the two machines?

The large discrepancy was dealt with by doing a full rescan of the network share. The remaining differences are those I reported subsequently, viz:
Linux 4,449 albums with 65,721 songs
Windows 4,452 albums with 65,795 songs


Why should a scan across a network share yield a different result - if networking was that unreliable corporate networks would be unworkable. The files are wither there or they aren't. The bug / fault must lie in the software?

Version info as follows:
6.5.2 - 12047 - Windows Server 2003 - EN - cp1252
6.5.3 - 12232 - Debian - EN - iso-8859-1

Guess I'll update the Windows instance, do a full rescan and see whether there are any remaining differences.

JJZolx
2007-07-03, 17:53
6.5.2 - 12047 - Windows Server 2003 - EN - cp1252
6.5.3 - 12232 - Debian - EN - iso-8859-1

Guess I'll update the Windows instance, do a full rescan and see whether there are any remaining differences.

Yeah, apples and oranges and all that.

mherger
2007-07-03, 23:26
> Why should a scan across a network share yield a different result - if
> networking was that unreliable corporate networks would be unworkable.

I'm seeing permission issues in our corporate network on a regular basis.
But this won't help you.

You should try to run a scan with a d_scan and d_info turned on. This
should spit out a list of what's going on with your files - which can be a
huge list. But it might have some indicators to why it's failing on some
files.

And with your number of files - can you really be sure there's no such
thing as a playlist, cuesheet, shortcut or anything. Could you please
doublecheck, just to be sure?

> The files are wither there or they aren't. The bug / fault must lie in
> the software?

As always. But there's a lot of software involved: networking stack,
database, scanning routine, operating systems...

--

Michael

-----------------------------------------------------------------
http://www.herger.net/SlimCD - your SlimServer on a CD
http://www.herger.net/slim - AlbumReview, Biography, MusicInfoSCR

Robin Bowes
2007-07-05, 16:30
egd wrote:

> Version info as follows:
> 6.5.2 - 12047 - Windows Server 2003 - EN - cp1252
> 6.5.3 - 12232 - Debian - EN - iso-8859-1
>
> Guess I'll update the Windows instance, do a full rescan and see
> whether there are any remaining differences.

Yeah, that's probably the root cause, i.e. differences in the code
between 6.5.2 and 6.5.3.

I must do the same test with my library sometime...

R.

egd
2007-07-06, 01:55
You should try to run a scan with a d_scan and d_info turned on. This should spit out a list of what's going on with your files - which can be a huge list. But it might have some indicators to why it's failing on some files.

Will do this when I have some time and report the results here.


And with your number of files - can you really be sure there's no such
thing as a playlist, cuesheet, shortcut or anything. Could you please doublecheck, just to be sure?

I'm certain. I have used Total Commander on a number of occasions to search for all of the above [*.pls *.m3u *.cue *.lnk]. All my cue files are renamed *.zzz. There are no shortcuts, playlists or anything else of the sort.


As always. But there's a lot of software involved: networking stack,
database, scanning routine, operating systems...


Agreed, but I doubt the issue lies in the network, dbms or os layers.