Plugin to test database tweaks

[PasTim]

Not quite - 7.9.0 - 1399470368 @ Wed May 7 14:26:02 UTC 2014 (as I said in my post).

If fixed after that I'll install the latest and test again in a day or 2.

I installed version 7.9.0 - 1399649268 @ Sat May 10 04:05:56 UTC 2014 and is it now OK, so I obviously picked up a version just before you fixed this problem.

All working well without me having to muck about manually, and without the tweak plugin. Good stuff

[Wirrunna]

Today I downloaded 7.9.0 - 1399649268 @ Sat May 10 and installed it onto my Win 7 i3-4130 workstation and the recently replaced (by VortexBox) Win 7 AMD E-350 1.6GHz HTPC. The music libraries were synced with the VortexBox, giving me 3 systems of varying power with their own local disks containing the same music library. The two Win boxes have 2 x Samsung 2TB drives, the VortexBox has a 4TB Seagate drive for music. All have SSD drives for the system disk. The Win boxes have had the RAMDisk removed.

I ran clear and rescans on each system and then a bunch of lookups - New Music, search artist for "the girl", album browse and artist browse.

The AMD E-350 system performs so well I need not have gone and replaced it by the VortexBoxes. New Music is almost instant, browse by Artists can take a second or two after selecting a letter and the artist picture is a bit slow but not unacceptable. Browse by Albums takes 2 to 3 seconds per change of letter for the albums to appear, but again, not unacceptable.
The i3 powered workstation is a bit quicker but not twice as fast.
The VortexBox running 7.8 with the Tweak DB plugin installed and set to 500 is the fastest.

Clear & rescan times show the great improvements that have been made with LMS 7.9
The differences between systems were as expected with the exception of MusicIP Import and Artist Picture lookup - the Artist Picture lookup shows there is something seriously wrong with either my setup or the way this function works in Windows.

[Wirrunna]

Gross speedup of Artist Picture Lookup in Windows

Logitech Media Server Version: 7.9.0 - 1399649268 @ Sat May 10 03:05:32 CUT 2014
Hostname: Daisy

Artist picture lookup (3135 of 3135) Complete 01:58:39

Logitech Media Server Version: 7.9.0 - 1399988233 @ Tue May 13 14:08:10 CUT 2014
Hostname: Daisy

Artist picture lookup (3136 of 3136) Complete 00:30:29

Took a quarter of the previous time - well done Michael. Will test further.

[mherger]

Plugin to test database tweaks

> Artist picture lookup (3136 of 3136) Complete 00:30:29
>
> Took a quarter of the previous time - well done Michael. Will test
> further.

Are you sure it's not due to the fact that the first time it had to
download stuff from the internet, while the second time around it was
able to pick up stuff from the cache? I wouldn't know what particular
change would have made such a dramatic difference...

--

Michael

[Wirrunna]

>
Are you sure it's not due to the fact that the first time it had to
download stuff from the internet, while the second time around it was
able to pick up stuff from the cache? I wouldn't know what particular
change would have made such a dramatic difference...
Michael

The second scan was from a New and Changed, so after seeing your post I power cycled Daisy and started a Clear & Rescan -
Artist picture lookup (3136 of 3136) Complete 00:49:50
Still a hell of an improvement.

I save the artist pictures to a directory and do not ever fiddle with or delete them.

I was hoping that " Speed up web requests for artwork, stylesheets etc. by not passing it to the templating engine." which was committed on 10/5 might have had something to do with the speed up.

Take the credit Michael and hope that it sticks around !

[PasTim]

Just to let you know, using the "7.9.0 - 1399649268 @ Sat May 10 04:05:56 UTC 2014" update, without the tweak plugin, I have recently found that a couple of my custom scan menus are still slow (several tens of seconds) and busy enough to interrupt playing music the 1st time they are run after a restart, unless I do a full vacuum on persist.db after a rescan. It isn't too onerous to do this just once after each rescan, so it's not a problem for me.

[mherger]

Plugin to test database tweaks

> Just to let you know, using the "7.9.0 - 1399649268 @ Sat May 10
> 04:05:56 UTC 2014" update, without the tweak plugin, I have recently
> found that a couple of my custom scan menus are still slow (several tens
> of seconds) and busy enough to interrupt playing music the 1st time they
> are run after a restart, unless I do a full vacuum on persist.db after
> a rescan. It isn't too onerous to do this just once after each rescan,
> so it's not a problem for me.

That's quite surprising, as vacuum now is part of the optimize step at
the end of every scan.

--

Michael

[JJZolx]

(sorry, wrong thread)

[PasTim]

> Just to let you know, using the "7.9.0 - 1399649268 @ Sat May 10
> 04:05:56 UTC 2014" update, without the tweak plugin, I have recently
> found that a couple of my custom scan menus are still slow (several tens
> of seconds) and busy enough to interrupt playing music the 1st time they
> are run after a restart, unless I do a full vacuum on persist.db after
> a rescan. It isn't too onerous to do this just once after each rescan,
> so it's not a problem for me.

That's quite surprising, as vacuum now is part of the optimize step at
the end of every scan.

Michael

But presumably not at the end of a Custom Scan?

When I vacuum, I stop LMS before running the vacuum using a sqlite3 command. I don't know if this makes any difference.

[JJZolx]

Now that the SQLite memory setting has been moved into Settings > Advanced > Performance, I see the following popup warning when changing the setting:

"Changes will take place at the next application restart for the following plugins:"

and nothing listed.

Is this message necessary, or is the memory setting made on the fly? If it is necessary, then shouldn't it be the same message shown when plugins are enabled/disabled? "Please restart Logitech Media Server for the changes to take effect."

[mherger]

Plugin to test database tweaks

> Now that the SQLite memory setting has been moved into Settings >
> Advanced > Performance, I see the following popup warning when changing
> the setting:
>
> "Changes will take place at the next application restart for the
> following plugins:"

Sounds like the wrong text to me anyway :-). I think that message is
wrong or at least not crucial in most cases. Changing that parameter
required a restart with MySQL. With SQLite not all changes will be made
unless you restart, but the most important (buffer size) will be applied
anyway. I'll review that. Thanks!

--

Michael

[stevedresden]

What happened to this plugin? It's now commented out in the test.xml file. Does it no longer work with 7.9?

[mherger]

Plugin to test database tweaks

> What happened to this plugin? It's now commented out in the test.xml
> file. Does it no longer work with 7.9?

The insights we gathered using it are now built in to 7.9. What exactly
are you looking for?

--

Michael

[mamema]

> What happened to this plugin? It's now commented out in the test.xml
> file. Does it no longer work with 7.9?

The insights we gathered using it are now built in to 7.9. What exactly
are you looking for?

--

Michael

sorry, for digging out this old thred, but i have extreme problems with sql queries from custom browse menues. I have 80.000 tracks and some menues are stuck for 5 (!) minutes and the music stops.

As i have enough power (4 core, 2.1 Ghz, 32 GB RAM) i would ask for tweaks to speed up the sql queries.

[erland]

sorry, for digging out this old thred, but i have extreme problems with sql queries from custom browse menues. I have 80.000 tracks and some menues are stuck for 5 (!) minutes and the music stops.

As i have enough power (4 core, 2.1 Ghz, 32 GB RAM) i would ask for tweaks to speed up the sql queries.

Would you be willing to install Database Query plugin and run the “Custom Scan Statistics” query to give us a better understanding of your Custom Scan data.
It would also help to get more information about which SQL statements or Custom Browse menu levels that are extremely slow.

I suspect the issue is that Custom Scan/Browse isn’t optimized for really large libraries but it’s hard to say for sure without more information regarding your setup.