Scanning library much slower on 7.4?

[JJZolx]

Note that scan time can vary greatly depending on the types of files you have. For example, FLAC is now very fast in 7.4.1, but MP3 is not quite as fast. If you have embedded artwork in your files, that slows it down a bit, etc. If you have 16MB embedded cover art (I won't name names you can expect a rather slow scan. So you can't base any speed claims solely on the number of files you have.

What happens in the case of _both_ embedded art and separate cover/folder.jpg files? Is the embedded artwork still extracted? And if you embed each album cover image in every track of an album, does that mean the scanner will extract them all and creates its resized versions (I think there are now five or more sizes created) for every one?

[andyg]

What happens in the case of _both_ embedded art and separate cover/folder.jpg files? Is the embedded artwork still extracted? And if you embed each album cover image in every track of an album, does that mean the scanner will extract them all and creates its resized versions (I think there are now five or more sizes created) for every one?

Currently the way it works is:

During the main scan phase, if embedded artwork is found, the track is flagged as having embedded artwork (cover=1 in the db). The album cover is set to point to the artwork for the first track scanned for that album. No check is made for cover.jpg/folder.jpg/etc.

If no embedded artwork is found for any track in an album, we make another pass (the artwork phase) looking for standalone cover.jpg/folder.jpg/etc files. If found, one track in the database gets the cover field set to the path to this file, and the album cover points to this track. These are then pre-cached into 5 sizes (unless pre-caching is disabled): thumb size pref or 100x100, 50x50, 40x40, 41x41, 64x64

One bug I've just found is that the embedded artwork is never pre-cached, I'm actually fixing this right now and changing things a bit so there is a pre-caching phase of the scanner.

I'm also changing the way artwork URLs are generated and cached so they are not based on track ID and correctly handle updated artwork, rescans, etc., to fix the current problem of old or mis-matched artwork appearing in the web UI after a rescan.

[MrSinatra]

That seems slow. My server is comparably spec'd (P4 3.0GH, 2 GB RAM), and 25k tracks take only about 27 minutes to fully scan.

Differences:
- my library consists of FLAC files
- I'm using an internal 7200 RPM SATA drive
- database configuration tweaks

Give more memory to MySQL and see what happens. I'm not sure if MP3s are slower to scan than FLAC, but I can test it easily with my MP3 mirror library.

i only have about 20 flacs or so. maybe one or two wma's. the rest is CBR 256kbps mp3.

the artwork is in the folder, not embedded. id3v2.3 tags only. no errors in the scanner log unless i'm missing it.

the music is on the ext usb2 raid1, but the system drive (where SBS is) is on a 36gig 10000rpm raptor.

i have not yet attempted moonbases (or anyone elses) DB tweaks. how did you arrive at the tweak settings you currently use?

i'm also not sure what other standard SBS / settings matter? guess tag formats? etc?

sounds like andy is dong some good stuff to artwork, however i am confused as to why these 5 sizes are used:

100x100, 50x50, 40x40, 41x41, 64x64

first, do you really need a 40x40 AND a 41x41???

second, where are they all used? they all seem small. i set 170x170 in the SBS settings for webui, so is SBS smart enough to know to make a 170x170 pre-cache version too?

[andyg]

Scanning library much slower on7.4?

On Oct 23, 2009, at 8:47 PM, MrSinatra wrote:

>
> the music is on the ext usb2 raid1, but the system drive (where SBS
> is)
> is on a 36gig 10000rpm raptor.

USB2 is a potential bottleneck, you'll get better performance with a
local disk obviously.

> 100x100, 50x50, 40x40, 41x41, 64x64
>
> first, do you really need a 40x40 AND a 41x41???

Heh indeed, these sizes are used on Controller and Radio's browse
lists. Unfortunately the different UIs have different sized artwork.

> second, where they all used? they all seem small. i set 170x170 in
> the SBS settings for webui, so is SBS smart enough to know to make a
> 170x170 pre-cache version too?

Yeah in your case 170x170 is made instead of 100x100. The first size
is the thumbSize pref.

[mherger]

Scanning library much slower on7.4?

> What happens in the case of _both_ embedded art and separate
> cover/folder.jpg files?

Only the embedded art is being used then.

[MrSinatra]

USB2 is a potential bottleneck, you'll get better performance with a
local disk obviously.

you would think so, and prob true, but my scans pre new scanner were with a local disk, ata133 i think, and it took basically an hour then too, but perhaps with somewhat fewer files.

i could try some internals for testing purposes, but i'm not sure what rpm they are.

> first, do you really need a 40x40 AND a 41x41???

Heh indeed, these sizes are used on Controller and Radio's browse
lists. Unfortunately the different UIs have different sized artwork.

i appreciate your answers, but surely designers there can adjust one or the other UI so as not to require a one pixel differentiation file creation? surely this is silly?

one less art to make = quicker scan time. and actually, i only use webui and the SBC, so perhaps the user should be able to pick via option which artworks actually need to be created given their usage?

[Gregorykent]

slow scan

Hello all,
I just installed 7.4 last evening and the directory scan took over 10 hours and the artwork scan is still running. I am running XPSP3, my storage is a WD Worldbook network drive cabled to a Netgear Wireless Router. The stats indicate some 20K items but previously I had some 40K--whole folders are missing from the SBS that are on the drive. The folder on the drive is assigned a drive letter via the map network drive function and SBS is directed to that. The use iTunes box was left unchecked. Within my music folder are folders I moved from iTunes, which I no longer use. Does the SBS recognize the folders as iTunes material and not scan them?

Other questions--can I just turn off the artwork business altogether? I never wanted it--it just shows up when the individual artist folders have the embedded artwork files. And is there a faster way to scan?--these times seem way beyond even what others are reporting here. The older 7.3.x took about 6 hours.

Thanks for any help.

[MrSinatra]

That seems slow. My server is comparably spec'd (P4 3.0GH, 2 GB RAM), and 25k tracks take only about 27 minutes to fully scan.

Differences:
- my library consists of FLAC files
- I'm using an internal 7200 RPM SATA drive
- database configuration tweaks

Give more memory to MySQL and see what happens. I'm not sure if MP3s are slower to scan than FLAC, but I can test it easily with my MP3 mirror library.

so here are my new stats. slightly newer SBS, and with moonbases file, but not without some big problems, like cranking hard drive (after the scan "finished") and less responsive [reactive?] webui:

Version: 7.4.2 - r29001 @ Sun Oct 25 06:02:26 PDT 2009
Hostname: kestrelrock
Server IP Address: 192.168.0.101
Server HTTP Port Number: 9000
Operating system: Windows XP - EN - cp1252
Platform Architecture: 586
Perl Version: 5.10.0 - MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt-log
Total Players Recognized: 1

Library StatisticsTotal Tracks: 34,355
Total Albums: 2,757
Total Artists: 912
Total Genres: 154
Total Playing Time: 2403:23:19

Music Scan DetailsDirectory Scan (34355 of 34355) Complete 00:40:32
Playlist Scan (5 of 5) Complete 00:00:00
Merge Various Artists (2757 of 2757) Complete 00:00:21
Artwork Scan (2757 of 2757) Complete 00:11:08

The server has finished scanning your music collection.
Total Time: 00:52:01 (Sunday, October 25, 2009 / 17.12)

http://forums.slimdevices.com/showpost.php?p=477453&postcount=73