Doubling of scan time moving from 7.6.x to LMS

[audiomuze]

Last weekend I installed Ubuntu Server 12.04 64 bit on a new HP Microserver with a faster CPU than my earlier Microserver (1.3Ghz vs 1.5Ghz) and moved the drivies housing my collection to this server. I configured LMS in exactly the same way it was configured on the previous machine and proceeded to do a full scan (seeing as there are changes to the underlying DB I had no choice). In any event, what used to take around 5 hours (yes, it's a large collection) on my slower server running 7.6.x took over 12 hours on LMS - and I made sure LMS was set to scan only for audio. What the hell has happened to the code that the scanner routine now needs more than 2x the time using a CPU that's 15% faster and translates to a server that's around 11% better across all performance measures using the same drives?

[garym]

Last weekend I installed Ubuntu Server 12.04 64 bit on a new HP Microserver with a faster CPU than my earlier Microserver (1.3Ghz vs 1.5Ghz) and moved the drivies housing my collection to this server. I configured LMS in exactly the same way it was configured on the previous machine and proceeded to do a full scan (seeing as there are changes to the underlying DB I had no choice). In any event, what used to take around 5 hours (yes, it's a large collection) on my slower server running 7.6.x took over 12 hours on LMS - and I made sure LMS was set to scan only for audio. What the hell has happened to the code that the scanner routine now needs more than 2x the time using a CPU that's 15% faster and translates to a server that's around 11% better across all performance measures using the same drives?

no help, other than recalling some older threads on the issue of large libraries and slower scanning....

http://forums.slimdevices.com/showth...nning+with+lms
http://forums.slimdevices.com/showth...nning+with+lms

[mlsstl]

I'd still suggest looking for a hardware, OS or LMS configuration error somewhere.

I recently moved to a Foxcon Atom server with 2 GB RAM and a scan of my 53,000 item collection with artwork took 47 minutes. That is about 15% faster than my prior Celeron PC with 1 GB memory. I now run Vortexbox software with LMS 7.7.2 and the prior machine was an ordinary Fedora Core 14 with the LMS updated routinely from the old SBS software.

I've seen nothing in any of the recent versions of the Squeezebox server software that has slowed things down.

[audiomuze]

I'd still suggest looking for a hardware, OS or LMS configuration error somewhere.

I recently moved to a Foxcon Atom server with 2 GB RAM and a scan of my 53,000 item collection with artwork took 47 minutes. That is about 15% faster than my prior Celeron PC with 1 GB memory. I now run Vortexbox software with LMS 7.7.2 and the prior machine was an ordinary Fedora Core 14 with the LMS updated routinely from the old SBS software.

I've seen nothing in any of the recent versions of the Squeezebox server software that has slowed things down.

There's no hardware error or anything of the sort going on here. Like you, I'm using faster hardware than was previously the case and a full scan has become ludicrous. What used to take around 5 hours now takes just over double time for the same library:

Version: 7.6.2 - r33593 @ Tue Nov 1 02:02:09 PDT 2011
Hostname: audioserv
Server IP Address: 127.0.0.1
Server HTTP Port Number: 9000
Operating system: Debian - EN - utf8
Platform Architecture: x86_64-linux
Perl Version: 5.10.1 - x86_64-linux-gnu-thread-multi
Database Version: DBD::SQLite 1.34_01 (sqlite 3.7.7.1)

Total Tracks: 203,274
Total Albums: 13,347
Total Artists: 4,022
Total Genres: 676
Total Playing Time: 14349:56:09

Music Scan Details
Discovering files/directories: /diskpool/d1/audiolib (216861 of 216861) Complete 00:06:04
Scanning new files: /diskpool/d1/audiolib (203281 of 203281) Complete 10:41:36
Discovering playlists: /tmp (1 of 1) Complete 00:00:00
Pre-caching Artwork (12803 of 12803) Complete 00:44:20
The server has finished scanning your music collection.
Total Time: 11:32:00 (Saturday, June 2, 2012 / 10:46 PM)

WTH? Scanner priority is set to highest. How on earth can this be considered reasonable?

[JJZolx]

I get very similar scan times in 7.7 and 7.8 as I do in 7.6.

You can easily install 7.6 if you like, even if only to see whether or not it makes any difference. Download it from the URL below. The 'unreleased' versions of 7.5 and 7.6 are the very last updates made to those branches, with all of the minor bug fixes that were checked in.

http://downloads.slimdevices.com

[JJZolx]

What used to take around 5 hours now takes just over double time for the same library:

Version: 7.6.2 - r33593 @ Tue Nov 1 02:02:09 PDT 2011

Wait a minute... You previously said LMS (version 7.7) was slower than 7.6. But you've posted what you call slow results from running Squeezebox Server (version 7.6.2), which you had said was faster on the old hardware.

Clearly, you can't logically blame the Squeezebox software.

[JJZolx]

Are you sure you weren't running 7.5 on the old server?

• 7.5 = Squeezebox Server, with MySQL as the database engine.
  
  The MySQL database server is much more configurable than SQLite, so on systems with more memory, you can increase MySQL performance substantially over the stock configuration.
  
  
• 7.6 = Squeezebox Server, with SQLite as the database engine.
  
  Faster for most users (scanning is often double the speed) than 7.5 with MySQL, but some people with very large libraries have reported that it's slower. A full scan of my 40k track library is considerably faster in 7.6 than it was in 7.5 with MySQL tweaked to the max.
  
  
• 7.7 = Logitech Media Server (LMS), using SQLite.
  
  This ads a DLNA media server to the old music-only server. If you're running a music-only server, it's good to double check the library configuration to make sure that no folders are being scanned for photos or videos. The DLNA server itself can also be completely disabled by disabling the plugin that implements it.
  
  
• 7.8 = Logitech Media Server (LMS), using SQLite.
  
  Next major release. At the moment, very little difference from 7.7.

[audiomuze]

Wait a minute... You previously said LMS (version 7.7) was slower than 7.6. But you've posted what you call slow results from running Squeezebox Server (version 7.6.2), which you had said was faster on the old hardware.

Clearly, you can't logically blame the Squeezebox software.

Why not? The results I posted were from a fresh install and scan carried out yesterday. The scan time was better than anything I've achieved with LMS. I've not run any versions of SBS/LMS that uses MySQL for a very, very long time now. As to no changes to scanning logic from SBS to LMS I disagree - in LMS I've had an artist listed as "-", browsing the artist link showed no underlying albums or anything of the sort. Same music collection, no tag changes, scanned with SBS build that yielded the above results and there's no such artist...

I've been a Squeezebox user since around 2004 and would like to believe I've a fairly good grip on how it should be configured, what underlying DB is in use etc. and how it typically performs in my Microserver. The recent builds have resulted in at least a doubling of scan times relative to earlier builds using SQLite. I'd specifically ensured that LMS was set to only scan music files and the SBS build I'm now using (Version: 7.6.2 - r33593 @ Tue Nov 1 02:02:09 PDT 2011) doesn't include video scanning options. Whatever the earlier build was that I was running managed to do a full scan on the same library in around 5.5 hours. Perhaps you're right though - I may have bren using some 7.5 build as I don't typically ugrade often.

The non-LMS thing that could be in play here is that the new server is running with only a single memory bank populated with 2GB of RAM whereas the earlier Microserver had both memory banks populated, albeit with two 1GB memory chips - it may be that the lack of Dual-channel memory is compromising the faster Microserver's performance.

[JJZolx]

Why not? The results I posted were from a fresh install and scan carried out yesterday. The scan time was better than anything I've achieved with LMS.

You're not making any sense. From your first post:

In any event, what used to take around 5 hours (yes, it's a large collection) on my slower server running 7.6.x took over 12 hours on LMS.

And then you posted that a 7.6.2 scan on the new server took 11.5 hours to complete. So where do you come to the conclusion that LMS 7.7 is taking twice as long as 7.6.2?

Whatever the earlier build was that I was running managed to do a full scan on the same library in around 5.5 hours.

Same library on the _old_ server. Have you ever had a 5.5 hour scan of the library on the _new_ server that you've built?

[Mnyb]

The dlna/media scan code is a bit flaky imho .

I disable it at startup in /etc/sysconfig/squeezeboxserver

with these options --noimage --novideo --noupnp then I found no big difference with previus version , I'm on 7.8 wich is almost the same as 7.7.3 at the moment .

Note: I'm using the rpm build the name change is not complete there both user and app are squeezeboxserver even if it is LMS .

# Edit this to suit your setup
SQUEEZEBOX_USER="squeezeboxserver"
SQUEEZEBOX_HOME="/usr/libexec"
SQUEEZEBOX_CFG_DIR="/var/lib/squeezeboxserver/prefs"
SQUEEZEBOX_LOG_DIR="/var/log/squeezeboxserver"
SQUEEZEBOX_CACHE_DIR="/var/lib/squeezeboxserver/cache"
SQUEEZEBOX_CHARSET="utf8"
SQUEEZEBOX_ARGS="--daemon --noimage --novideo --noupnp --nosb1slimp3sync --prefsdir=$SQUEEZEBOX_CFG_DIR --logdir=$SQUEEZEBOX_LOG_DIR --cachedir=$SQUEEZEBOX_CACHE_DIR --charset=$SQUEEZEBOX_CHARSET"