Plugin to test database tweaks

[Mnyb]

Fair enough.

However, without custom tags I really couldn't use LMS, so I don't feel I have much choice. The people (Microsoft?) who invented the standard tags seem to have had no idea of the various types of music and their requirements

Schedule the scans when not listening , You can live without you latest tag change a day or two

[PasTim]

Schedule the scans when not listening , You can live without you latest tag change a day or two

I do I kick them off when I go to bed, and then Server Power Control shuts the system down later on.

[mherger]

Plugin to test database tweaks

> Do you turn on auto_vacuum during the scan, do a vacuum before scanning,
> or do a vacuum after scanning? Why not also do the remaining databases?

auto_vacuum is off for these two files. The auto_vacuum is mostly about
not wasting disk space, but it doesn't help the fragmentation. If you
want best performance, then you'll have to run a full VACUUM. As these
two files don't see that many changes the waste of disk space should be
negligible.

It's a different story for the cache files which change every minute if
you listen to some online source. This is the reason why I did not
enable auto_vacuum on library and persist, but on the cache files.

Initially I set vacuum to be run right after the scanner had wiped all
tables, as I expected this to be the quickest. Unfortunately the DB is
slightly slower if it has to constantly grow the file. Plus at the end
of the scan there would be a certain amount of fragmentation already.
Therefore I decided to run it in the optimize step.

> The thing I dislike about this approach is that if everything in LMS
> worked as it should (i.e. if there were no scanning bugs or bugs that
> cause the database to become inconsistent or corrupted) users should
> never have to run a full rescan.

We still have a long way to go :-).

> Having to do so to maximize performance
> is not the best strategy, IMO. For the users for whom this will do the
> most good, that full rescan may take hours due to very large libraries
> or very slow hardware.

Another change I just committed: the VACUUM is read during a new &
changed scan, too, if no player is active. The problem is that VACUUM
can break playback. If you use the rescan plugin to run scans at night,
you should be on the save side.

> Is the max buffer size per SQLite database or is there just one buffer
> for the entire LMS application?

It's per database connection, to be accurate. Eg. the scanner can use a
different buffer size than the server (which makes sense, as the scanner
doesn't profit as much from the larger buffer to its mostly writing nature).

Currently only persist and library use that maximum setting. Caches
don't. I probably have to tweak this further, as caches are accessed
quite often, while persist is rather simple (unless you're using one of
Erland's plugins). But I'd like to get some data from you first.

Keep in mind that in most cases LMS would be using a fraction of that
maximum cache_size only anyway. Eg. my server process is currently
running at around 200MB memory usage. Before the buffer change it was
around 140MB. It's far from using 500MB times two cache_size.
Unfortunately I haven't figured out a way yet to determine how much
sqlite is actually using.

--

Michael

[mherger]

Plugin to test database tweaks

> My question is as follows. I removed the tweak plugin whilst the
> indices had been cut back. I then installed the latest 7.9 and did a
> full scan. Will this recreate any missing indices or not?

No, it does not re-create them. The only way to do so without the plugin
is to delete the library.db file, which would cause LMS to start from
scratch.

--

Michael

[erland]

> The thing I dislike about this approach is that if everything in LMS
> worked as it should (i.e. if there were no scanning bugs or bugs that
> cause the database to become inconsistent or corrupted) users should
> never have to run a full rescan.

We still have a long way to go :-).

Just keep in mind that there might be more important to improve browsing functionality than improve scanning speed :-)
Especially since scanning speed already is acceptable for at least 95% of all LMS users, IMHO.

Currently only persist and library use that maximum setting. Caches
don't. I probably have to tweak this further, as caches are accessed
quite often, while persist is rather simple (unless you're using one of
Erland's plugins). But I'd like to get some data from you first.

I plan to experiment a bit with reordering the WHERE criterias in my plugins in case SQLite works as you indicated earlier in the thread and its optimiser prefers if the most limiting criteria comes first in the WHERE statement. Just have to be able to find some time to experiment with it...
Not sure if this affects anything regarding the buffer size choices you offer, I'm just mentioning it in case it does.

[Wirrunna]

OK, back down the coast after a few days at the farm, VPN'd into my Sydney server as I had left it doing a Clear & rescan.

7.8.0 with Tweak DB.
Logitech Media Server Version: 7.8.0 - 1395409907 @ Thu Mar 27 20:19:17 CUT 2014
Hostname: Gladstone
Total Tracks: 144,947
Total Albums: 11,302
Total Artists: 3,133
Total Genres: 104
Total Playing Time: 10038:39:14
Clear & Rescan
Discovering files/directories: F:\Music 1 (73783 of 73783) Complete 00:00:57
Scanning new music files: F:\Music 1 (66439 of 66439) Complete 00:28:54
Discovering files/directories: G:\Music 2 (86041 of 86041) Complete 00:01:25
Scanning new music files: G:\Music 2 (78509 of 78509) Complete 01:13:21
Discovering playlists: C:\Users\Johno\Music\Playlists (14 of 14) Complete 00:00:00
Scanning new playlists: C:\Users\Johno\Music\Playlists (13 of 13) Complete 00:00:08
MusicIP Import (144949 of 144949) Complete 00:19:16
Album cover lookup (8 of 8) Complete 00:00:15
Artist picture lookup (3134 of 3134) Complete 01:49:05
Pre-caching Artwork (11182 of 11182) Complete 00:05:46
The server has finished scanning your media library.

Bugger, just noticed I didn't get the total time ! - Added it up to 3:59:07


Removed Tweak DB plugin, upgraded to LMS 7.9.0 , Settings / Performance / DB Mem Config set to Max.
Logitech Media Server Version: 7.9.0 - 1399474655 @ Thu May 8 03:04:57 CUT 2014
Hostname: Gladstone
Memory use after startup
SQUEEZ~3.EXE 0 - 173,128 - 192,900 - 22,252 - 170,648
After a few lookups
SQUEEZ~3.EXE 0 - 547,800 - 569,108 - 24,576 - 544,532

Now for a clear & rescan.
SQUEEZ~3.EXE 0 - 417,108 - 439,948 - 26,092 - 413,856

Clear & Rescan
Discovering files/directories: F:\Music 1 (73783 of 73783) Complete 00:01:15
Scanning new music files: F:\Music 1 (66439 of 66439) Complete 00:25:03
Discovering files/directories: G:\Music 2 (86041 of 86041) Complete 00:01:43
Scanning new music files: G:\Music 2 (78509 of 78509) Complete 00:30:59
Discovering playlists: C:\Users\Johno\Music\Playlists (14 of 14) Complete 00:00:00
Scanning new playlists: C:\Users\Johno\Music\Playlists (13 of 13) Complete 00:00:02
MusicIP Import (144949 of 144949) Complete 00:19:43
Album cover lookup (8 of 8) Complete 00:00:18
Artist picture lookup (3133 of 3133) Complete 01:37:14
Pre-caching Artwork (11182 of 11182) Complete 00:05:42
The server has finished scanning your media library.
Total Time: 03:01:59 (Friday, May 9, 2014 / 11:19 AM)

Saved about 1 hour, just have to get the Artist Picture Lookup going faster - but that is a topic for a different thread.
Also, my earlier observation that memory was not returned is wrong. The amount of memory used by the server goes up and down like a yo-yo - current memory allocation -

SQUEEZ~3.EXE 0 - 240,188 - 259,476 - 22,664 - 236,812

[JJZolx]

Quoted from a different thread...

>> One potential issue with the
>> logitechmediaserver-7.9.0-0.10.20140508git1399582156 update, I've just
>> noticed. Re-scan "new and changed" seems to be locking the web-gui out
>> of the database. Can't remember, but I don't think this was the previous
>> behaviour. Might have something to do with Michael's recent db
>> "optimisations".
>
> I definitely don't see this behavior on Windows. FWIW.

It could be the VACUUM step at the end of a scan. Depending on your
library size and I/O performance this can take a while, during which the
web UI is non-responsive. It should only be run if no player is active.
You might not have seen it if the step was skipped due to some player
playing music in the background.

Maybe rethink the idea of having a daily scheduled vacuum instead? How about replacing the 'Rescan Music Library' plugin with one that can both rescan the library and vacuum the dbs? Call it 'Server & Database Maintenance' or something similar. Other things could be added to the plugin as desired, such as clearing the server.log (I do this on a daily basis as well).

[mherger]

Plugin to test database tweaks

> Maybe rethink the idea of having a daily scheduled vacuum instead? How
> about replacing the 'Rescan Music Library' plugin with one that can both
> rescan the library and vacuum the dbs?

Vacuum already is part of the nightly scan (unless you let it scan while
you're listening to some music).

IMHO it should just work. Nobody should need to take care of it
manually. Very much like the ANALYZE task - which is even more important
than VACUUM! It's been there "forever", nobody needed a special button
to activate it.

--

Michael

[bwaldron]

I have been running the plugin for the past few days on LMS 7.8.1 (Windows 7) without noticing any issues. However, I had not done a rescan of any sort during that time. When I added a new album to my collection yesterday, the scan aborted. Tried a full clear+rescan, same abort. I installed the latest nightly (1398786621) and still have the problem -- I have to disable the Tweak Database plugin in order to run a rescan. If I try and do so while it is installed, I get this:

----------scanner.log----------
[14-05-09 08:46:00.1732] main::main (202) Starting Logitech Media Server scanner (v7.8.1, 1398786621, Thu May 8 03:04:59 CUT 2014) perl 5.014001
[14-05-09 08:46:02.5013] Slim::Schema::forceCommit (2130) Warning: Trying to commit transactions before DB is initialized!
[14-05-09 08:46:04.0013] Slim::bootstrap::tryModuleLoad (285) Warning: Module [Plugins::Vacuum::Plugin] failed to load:
Base class package "Slim::Web::Settings" is empty.
(Perhaps you need to 'use' the module which defines that package first,
or make that module available in @INC (@INC contains: C:\ProgramData\Squeezebox\Cache\InstalledPlugins\P lugins\MusicArtistInfo\lib C:\ProgramData\Squeezebox\Cache\InstalledPlugins C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14\MSWin32-x86-multi-thread C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14\MSWin32-x86-multi-thread\auto C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14.1\MSWin 32-x86-multi-thread C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14.1\MSWin 32-x86-multi-thread\auto C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\MSWin32-x86-multi-thread C:\PROGRA~1\SQUEEZ~1\server\lib C:\PROGRA~1\SQUEEZ~1\server\CPAN C:/PROGRA~1/SQUEEZ~1/server c:\temp\pdk-bwaldron-1396\ C:\PROGRA~1\SQUEEZ~1\server).
at C:\ProgramData\Squeezebox\Cache\InstalledPlugins/Plugins/Vacuum/Plugin.pm line 44
BEGIN failed--compilation aborted at C:\ProgramData\Squeezebox\Cache\InstalledPlugins/Plugins/Vacuum/Plugin.pm line 44.
Compilation failed in require at (eval 1002) line 2.
BEGIN failed--compilation aborted at (eval 1002) line 2.

[14-05-09 08:46:04.0233] Slim::Utils::PluginManager::load (323) Error: Couldn't load Plugins::Vacuum::Plugin
[14-05-09 08:46:04.0828] Slim::Music::Import::runImporter (488) Starting Slim::Media::MediaFolderScan scan
[14-05-09 08:46:04.0842] Slim::Utils::Scanner::Local::rescan (174) Discovering audio files in M:\MP3
[14-05-09 08:46:06.1211] main::main (329) Error: Failed when running main scan: [Undefined subroutine & called at /<C:\PROGRA~1\SQUEEZ~1\server\scanner.exe>HTTP/Headers.pm line 341.
]
[14-05-09 08:46:06.1222] main::main (330) Error: Skipping post-process & Not updating lastRescanTime!
[14-05-09 08:46:06.9388] HTTP::Headers::_basic_auth (341) Warning: Undefined subroutine & called at /<C:\PROGRA~1\SQUEEZ~1\server\scanner.exe>HTTP/Headers.pm line 341.

[mherger]

Plugin to test database tweaks

Thanks! I disabled the plugin to be used in the scanner. Nobody was able
to see any advantage using the larger buffers anyway.

0.6.1 should be available shortly.

Michael



Am 09.05.14 14:59, schrieb bwaldron:
>
> I have been running the plugin for the past few days on LMS 7.8.1
> (Windows 7) without noticing any issues. However, I had not done a
> rescan of any sort during that time. When I added a new album to my
> collection yesterday, the scan aborted. Tried a full clear+rescan, same
> abort. I installed the latest nightly (1398786621) and still have the
> problem -- I have to disable the Tweak Database plugin in order to run a
> rescan. If I try and do so while it is installed, I get this:
>
> ----------scanner.log----------
> [14-05-09 08:46:00.1732] main::main (202) Starting Logitech Media Server
> scanner (v7.8.1, 1398786621, Thu May 8 03:04:59 CUT 2014) perl
> 5.014001
> [14-05-09 08:46:02.5013] Slim::Schema::forceCommit (2130) Warning:
> Trying to commit transactions before DB is initialized!
> [14-05-09 08:46:04.0013] Slim::bootstrap::tryModuleLoad (285) Warning:
> Module [Plugins::Vacuum::Plugin] failed to load:
> Base class package "Slim::Web::Settings" is empty.
> (Perhaps you need to 'use' the module which defines that package
> first,
> or make that module available in @INC (@INC contains:
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins\P lugins\MusicArtistInfo\lib
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins
> C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14\MSWin32-x86-multi-thread
> C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14\MSWin32-x86-multi-thread\auto
> C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14.1\MSWin 32-x86-multi-thread
> C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\5.14.1\MSWin 32-x86-multi-thread\auto
> C:\PROGRA~1\SQUEEZ~1\server\CPAN\arch\MSWin32-x86-multi-thread
> C:\PROGRA~1\SQUEEZ~1\server\lib C:\PROGRA~1\SQUEEZ~1\server\CPAN
> C:/PROGRA~1/SQUEEZ~1/server c:\temp\pdk-bwaldron-1396\
> C:\PROGRA~1\SQUEEZ~1\server).
> at
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins/Plugins/Vacuum/Plugin.pm
> line 44
> BEGIN failed--compilation aborted at
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins/Plugins/Vacuum/Plugin.pm
> line 44.
> Compilation failed in require at (eval 1002) line 2.
> BEGIN failed--compilation aborted at (eval 1002) line 2.
>
> [14-05-09 08:46:04.0233] Slim::Utils::PluginManager::load (323) Error:
> Couldn't load Plugins::Vacuum::Plugin
> [14-05-09 08:46:04.0828] Slim::Music::Import::runImporter (488) Starting
> Slim::Media::MediaFolderScan scan
> [14-05-09 08:46:04.0842] Slim::Utils::Scanner::Local::rescan (174)
> Discovering audio files in M:\MP3
> [14-05-09 08:46:06.1211] main::main (329) Error: Failed when running
> main scan: [Undefined subroutine & called at
> /<C:\PROGRA~1\SQUEEZ~1\server\scanner.exe>HTTP/Headers.pm line 341.
> ]
> [14-05-09 08:46:06.1222] main::main (330) Error: Skipping post-process &
> Not updating lastRescanTime!
> [14-05-09 08:46:06.9388] HTTP::Headers::_basic_auth (341) Warning:
> Undefined subroutine & called at
> /<C:\PROGRA~1\SQUEEZ~1\server\scanner.exe>HTTP/Headers.pm line 341.
>
>
> ------------------------------------------------------------------------
> bwaldron's Profile: http://forums.slimdevices.com/member.php?userid=1073
> View this thread: http://forums.slimdevices.com/showthread.php?t=101469
>
>

[bwaldron]

All good now, thanks!

Thanks! I disabled the plugin to be used in the scanner. Nobody was able
to see any advantage using the larger buffers anyway.

0.6.1 should be available shortly.

[PasTim]

I have hit a problem with 7.9.0 - 1399470368 @ Wed May 7 14:26:02 UTC 2014. I decide to try testing a regular automated scan (not something I've done before), with server power control waking the PC up 10 minutes before the scan is due. The PC duly woke up at 8:35, with a scan scheduled for 8:45. I've now tried this twice, and both times it has looped doing nothing except chewing up CPU. The server log (there is nothing on the scanner log) is as below. I eventually aborted the scan.

If I kick off a scan manually it seems to be OK.

2014-05-10 08:14:56 squeezeboxserver_safe stopped.
2014-05-10 08:35:32 squeezeboxserver_safe started.
[14-05-10 08:35:36.4170] main::init (368) Starting Logitech Media Server (v7.9.0, 1399470368, Wed May  7 14:26:02 UTC 2014) perl 5.018002
[14-05-10 08:35:43.9159] Slim::Utils::Misc::msg (1311) Warning: [08:35:43.9151] defined(@array) is deprecated at /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/SrvrPowerCtrl/Util.pm line 69.
[14-05-10 08:35:43.9167] Slim::Utils::Misc::msg (1311) Warning: [08:35:43.9162] 	(Maybe you should just omit the defined()?)
[14-05-10 08:35:44.8631] Plugins::CustomScan::Scanner::initDatabase (96) CustomScan: Creating database tables
[14-05-10 08:35:44.8645] Plugins::CustomScan::Scanner::executeSQLFile (498) Executing SQL file /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/SQL/SQLite/dbcreate.sql
[14-05-10 08:35:44.8843] Plugins::CustomScan::Scanner::initDatabase (439) Creating indexes
[14-05-10 08:35:44.8858] Plugins::CustomScan::Scanner::executeSQLFile (498) Executing SQL file /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/SQL/SQLite/dbindex.sql
[14-05-10 08:35:44.8917] Plugins::CustomScan::Plugin::installHook (132) Installing Custom Scan hooks
[14-05-10 08:35:50.4549] Plugins::SrvrPowerCtrl::Settings::new (209) Server Power Control, version 20120427.163654, is running on sbs version 7.9.0 on unix:Debian, mac  
[14-05-10 08:35:50.5990] Plugins::TrackStat::Storage::refreshTracks (1237) TrackStat: Synchronizing TrackStat data, please wait...
[14-05-10 08:35:54.4575] Plugins::TrackStat::Storage::refreshTracks (1886) TrackStat: Synchronizing TrackStat data finished
[14-05-10 08:45:00.3608] Slim::Utils::Scanner::Local::rescan (174) Discovering audio files in /storage/music/flac
[14-05-10 09:16:29.0547] Slim::Utils::Scanner::Local::__ANON__ (267) Removing deleted audio files (0)
[14-05-10 09:16:29.0555] Slim::Utils::Scanner::Local::__ANON__ (345) Scanning new audio files (0)
[14-05-10 09:16:29.0563] Slim::Utils::Scanner::Local::__ANON__ (425) Rescanning changed audio files (0)
[14-05-10 09:16:29.0618] Slim::Utils::Scanner::Local::rescan (174) Discovering audio files in /storage/music/playlist
[14-05-10 09:16:29.1242] Plugins::SrvrPowerCtrl::Watchdog::OnWakeupWatchdog (647) Wakeup Call!!
[14-05-10 09:16:29.3061] Slim::Schema::Storage::throw_exception (122) Error: DBI Exception: DBD::SQLite::db do failed: table diskonly already exists [for Statement "
    		CREATE TEMPORARY TABLE diskonly AS 
				SELECT          DISTINCT(url) as url
				FROM            scanned_files
				WHERE           url NOT IN (
					SELECT url FROM tracks
					WHERE id <= 380181
				)
				AND             url LIKE 'file:///storage/music/playlist%'
				AND             filesize != 0
    	"]
[14-05-10 09:16:29.3074] Slim::Schema::Storage::throw_exception (122) Backtrace:

   frame 0: Slim::Utils::Log::logBacktrace (/usr/share/perl5/Slim/Schema/Storage.pm line 122)
   frame 1: Slim::Schema::Storage::throw_exception (/usr/share/squeezeboxserver/CPAN/DBIx/Class/Storage/DBI.pm line 1006)
   frame 2: DBIx::Class::Storage::DBI::__ANON__ (/usr/share/perl5/Slim/Utils/Scanner/Local.pm line 218)
   frame 3: Slim::Utils::Scanner::Local::__ANON__ (/usr/share/perl5/Slim/Networking/IO/Select.pm line 168)
   frame 4: (eval) (/usr/share/perl5/Slim/Networking/IO/Select.pm line 168)
   frame 5: (eval) (/usr/share/perl5/Slim/Networking/IO/Select.pm line 168)
   frame 6: Slim::Networking::IO::Select::loop (/usr/sbin/squeezeboxserver line 724)
   frame 7: main::idleStreams (/usr/share/perl5/Slim/Web/Template/Context.pm line 32)
   frame 8: Slim::Web::Template::Context::process (/usr/share/squeezeboxserver/CPAN/Template/Service.pm line 85)
   frame 9: (eval) (/usr/share/squeezeboxserver/CPAN/Template/Service.pm line 82)
   frame 10: Template::Service::process (/usr/share/squeezeboxserver/CPAN/Template.pm line 65)
   frame 11: Template::process (/usr/share/perl5/Slim/Web/Template/SkinManager.pm line 322)
   frame 12: Slim::Web::Template::SkinManager::_fillTemplate (/usr/share/perl5/Slim/Web/Template/NoWeb.pm line 68)
   frame 13: Slim::Web::Template::NoWeb::_generateContentFromFile (/usr/share/perl5/Slim/Web/HTTP.pm line 2548)
   frame 14: Slim::Web::HTTP::filltemplatefile (/usr/share/perl5/Slim/Web/Pages/Progress.pm line 132)
   frame 15: Slim::Web::Pages::Progress::progressBar (/usr/share/perl5/Slim/Web/Pages/Common.pm line 69)
   frame 16: Slim::Web::Pages::Common::addLibraryStats (/usr/share/perl5/Slim/Web/Pages/Home.pm line 149)
   frame 17: Slim::Web::Pages::Home::home (/usr/share/perl5/Slim/Web/HTTP.pm line 1096)
   frame 18: Slim::Web::HTTP::generateHTTPResponse (/usr/share/perl5/Slim/Web/HTTP.pm line 919)
   frame 19: Slim::Web::HTTP::processURL (/usr/share/perl5/Slim/Web/HTTP.pm line 723)
   frame 20: Slim::Web::HTTP::processHTTP (/usr/share/perl5/Slim/Networking/IO/Select.pm line 123)
   frame 21: (eval) (/usr/share/perl5/Slim/Networking/IO/Select.pm line 119)
   frame 22: Slim::Networking::IO::Select::__ANON__ (/usr/share/perl5/Slim/Networking/IO/Select.pm line 168)
   frame 23: (eval) (/usr/share/perl5/Slim/Networking/IO/Select.pm line 168)
   frame 24: Slim::Networking::IO::Select::loop (/usr/sbin/squeezeboxserver line 705)
   frame 25: main::idle (/usr/sbin/squeezeboxserver line 661)
   frame 26: main::main (/usr/sbin/squeezeboxserver line 1195)

[14-05-10 09:16:29.3087] Slim::Utils::Misc::msg (1311) Warning: [09:16:29.3081] EV: error in callback (ignoring): Carp::Clan::__ANON__(): DBI Exception: DBD::SQLite::db do failed: table diskonly already exists [for Statement "
    		CREATE TEMPORARY TABLE diskonly AS 
				SELECT          DISTINCT(url) as url
				FROM            scanned_files
				WHERE           url NOT IN (
					SELECT url FROM tracks
					WHERE id <= 380181
				)
				AND             url LIKE 'file:///storage/music/playlist%'
				AND             filesize != 0
    	"] at /usr/share/perl5/Slim/Schema/Storage.pm line 126

[sander]

OK I have been meaning to contribute, but was hoping to set up a better test environment. Alas the torrid pace of LMS development these days means I better get moving if I want to help.

I have 2 sets of data. Now my pre 7.73 version has sqlhelper.pm modified to give slightly more memory:
132 push @{$sql}, 'PRAGMA cache_size = 2000000';
so the results are skewed.

7.7.3 - "prod" system - Ubuntu 12.04.4 LTS 32-bit Virtualbox VM with 2GB of RAM
Logitech Media Server Version: 7.7.3 - 1375965195 @ Mon Aug 12 03:09:39 PDT 2013

Media Scan Details
Discovering files/directories: /c/media/Music (211634 of 211634) Complete 00:04:11
Scanning new music files: /c/media/Music (449 of 449) Complete 00:01:51
Scanning changed files: /c/media/Music (493 of 493) Complete 00:00:49
Discovering playlists: /c/media/playlists (61 of 61) Complete 00:00:00
Scanning changed playlists: /c/media/playlists (7 of 7) Complete 00:00:08
MusicIP Import (5246 of 5246) Complete 00:00:25
Profile export (930 of 930) Complete 00:00:07
Pre-caching Artwork (143 of 143) Complete 00:00:59
Total Time: 00:08:30

Output of Gordon Harris' get slim-slim-mem-usage.sh:
Rss usage: 436,256 All memory the process uses, including all memory this process shares with other processes
Shared usage: 1,216 Memory that this process shares with other processes
Private usage: 435,040 Private memory used by this process
Swap usage: 86,104 Swap memory used by the process
Pss usage: 435,122 Proportional Set Size, a good overall memory indicator
Size usage: 551,080 Virtual size, almost meaningless
Referenced usage: 347,232 Memory currently marked as referenced or accessed

Test system:
Centos 6.5 64-bit Virtualbox VM with 1GB of RAM
Logitech Media Server Version: 7.9.0 - 1399649268 @ Sat May 10 04:01:05 UTC 2014

Music scan details
Discovering files/directories: /c/media/Music (211884 of 211884) Complete 00:11:22
Removing deleted files: /c/media/Music (5 of 5) Complete 00:00:12
Scanning new music files: /c/media/Music (672 of 672) Complete 00:05:35
Scanning changed files: /c/media/Music (2210 of 2210) Complete 00:13:27
Discovering files/directories: /c/media/playlists (60 of 60) Complete 00:00:00
Scanning new music files: /c/media/playlists (4 of 4) Complete 00:00:05
Scanning changed files: /c/media/playlists (8 of 8) Complete 00:00:30
Find updated coverart files (12519 of 12519) Complete 00:02:42
Pre-caching Artwork (620 of 620) Complete 00:03:55
Total Time: 00:37:48

Output of Gordon Harris' get slim-slim-mem-usage.sh:
Rss usage: 605,960 All memory the process uses, including all memory this process shares with other processes
Shared usage: 764 Memory that this process shares with other processes
Private usage: 605,196 Private memory used by this process
Swap usage: 0 Swap memory used by the process
Pss usage: 605,301 Proportional Set Size, a good overall memory indicator
Size usage: 795,756 Virtual size, almost meaningless
Referenced usage: 521,452 Memory currently marked as referenced or accessed

A couple of caveats:
my 7.7.3 system sometimes hangs with a cpu-eating, uninterruptible perl script which causes me to have to reboot the server, so I don't update it that often. I can't remember when it started so my fix is to move to a new version. That said it is rock solid otherwise and is used for playback at least about 5 hours every day.
The test server has half the memory, no MusicIP, next to no trackstat activity, and has barely been used.

My pain point with the default memory settings was searches, browsing making the system unusable and browsing music folders just hung the system. With the old SQL hack it was good and with the new one it seems better.
The total scan time on the new version may be slower due to the lower memory on the vm which I have now raised to match the other version. I can do a complete rescan with both systems now if that helpful.

Thanks Michael for making this happen!

[mherger]

Plugin to test database tweaks

> [14-05-10 09:16:29.3061] Slim::Schema::Storage::throw_exception (122) Error: DBI Exception: DBD::SQLite::db do failed: table diskonly already exists [for Statement "
>

Are you running the latest build? I thought I had fixed this a few days ago.

[PasTim]

> [14-05-10 09:16:29.3061] Slim::Schema::Storage::throw_exception (122) Error: DBI Exception: DBD::SQLite::db do failed: table diskonly already exists [for Statement "
>

Are you running the latest build? I thought I had fixed this a few days ago.

Not quite - 7.9.0 - 1399470368 @ Wed May 7 14:26:02 UTC 2014 (as I said in my post).

If fixed after that I'll install the latest and test again in a day or 2.