PDA

View Full Version : Is it me or is the web interface getting slower on every new release?



docbee
2005-10-25, 14:27
Hi,
when connecting with a browser to slimserver 6.2 it takes up to 10 seconds until the page is there. I am sure that previous versions were much faster with this. my slimserver is running on a 800 MHz P3 with 256MB of RAM. It is running on linux, I have about 30.000 titles in the database.

Anyone else having this slow-down as well? Does album art have an impact on this? just guessing...

I appreciate the new features of slimserver 6.x, but I hate to have this at the price of dramatically slowing down the web interface.

Looks to me like the storage management of the slimserver application or the underlying perl itself doesn't handle the given amount of data too good. When I do the same things quite fast again and again on the web interface it significantly speeds up - for the moment. Very strange...

mherger
2005-10-25, 14:43
> when connecting with a browser to slimserver 6.2 it takes up to 10
> seconds until the page is there.

Is this with every page or only some of them? Do you use MusicMagic?
iTunes? Moodlogic? Please disable any of these plugins if you don't need
them.

This is just to say that 10 seconds is far from normal. I'm using a
comparable machine with good load times. Brows cover art is slower
compared to the others. But standard is well below 10 seconds.

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

kdf
2005-10-25, 14:47
Quoting docbee <docbee.1xh8rz (AT) no-mx (DOT) forums.slimdevices.com>:

>
> Hi,
> when connecting with a browser to slimserver 6.2 it takes up to 10
> seconds until the page is there. I am sure that previous versions were
> much faster with this. my slimserver is running on a 800 MHz P3 with
> 256MB of RAM. It is running on linux, I have about 30.000 titles in the
> database.

There are several sections of the code for all skins that has changed. The
server uses template toolkit to create full pages. These are cached but the
new templates would cause those to be flushed and restarted. After long idle
periods, the taks coudl be swapped out and you may experience a lag when
starting to use it again. In general use, I haven't personally noticed much
change either way.

>
> Anyone else having this slow-down as well? Does album art have an
> impact on this? just guessing...

It is more information to move around, so there is a cost.

> I appreciate the new features of slimserver 6.x, but I hate to have
> this at the price of dramatically slowing down the web interface.

"Dramatic" seems a bit unfair. In light of that, I do wonder if there is
something specific to your setup that is giving you lesser performance. Are
certain skins better for you than others? Is it worse for certain tasks than
others? Do you have any third party plugins installed?

-kdf

jlynch3
2005-10-25, 15:11
It seems you're running a bit light on the RAM side. Do you serve other web content from the box? How quick does that come down?

meyergru
2005-10-25, 16:45
Hi folks!


I can see the same behaviour as docbee: Slimserver 6.2 seems pretty slow for normal browser accesses. I have traced Slimserver with strace and found some strange things happening.

Even when I just browse to http://slimserver:9000 (not only on first occurrence), there are really long sequences (i.e. thousands of repetitions) of messages like the following in the strace log:

_llseek(8, 3961856, [3961856], SEEK_SET) = 0
read(8, "\r\0\0\0#\0j\0\3\346\3\311\3\263\3\234\3\206\3j\3Q \003"..., 1024) = 1024
_llseek(13, 400384, [400384], SEEK_SET) = 0
write(13, "\n\0\0\0\5\0L\0\0L\0\367\1\235\2q\3N\0\0\0\0\0\0\0 \0\0"..., 1024) = 1024
_llseek(13, 401408, [401408], SEEK_SET) = 0
write(13, "\n\0\0\0\5\0\206\0\0\206\1\21\1\365\2\177\3=\0\0\0 \0\0"..., 1024) = 1024
_llseek(8, 3962880, [3962880], SEEK_SET) = 0
read(8, "\r\3!\0\3\0\211\0\0\211\2[\1i\0A\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
_llseek(12, 235520, [235520], SEEK_SET) = 0
write(12, "\r\0\0\0\5\0t\0\3\23\2\205\1\334\1\24\0t\0\0\0\0\0 \0\0"..., 1024) = 1024
_llseek(12, 236544, [236544], SEEK_SET) = 0
write(12, "\r\0\0\0\5\0\254\0\3V\2\303\2F\1\224\0\254\0\0\0\0 \0\0"..., 1024) = 1024

The file handles are #8 = %CACHEDIR%/.slimserversql.db, #12 and #13 are temporary files generated by SQLite like /var/tmp/sqlite_%SOMETHING%.

Frankly, I don't quite understand this:

1. I doubt that this huge number of database read accesses (looks like a full title scan to me) are neccessary in the first place when I just want to look at the start page. The number of accesses to handle #8 is way larger than any potential number of songs shown on the start page (e.g. the active playlist).

2. Even if there is a reason for the read accesses, I cannot see why there are write accesses for every entry. This looks like a modify to each accessed record, since the two temp files seem to be short-lived transaction logs for SQLite?

Oh, and BTW: I have no plugins whatsoever but a rather large music archive (approx. 40000 songs). Slimserver takes around 45 MByte of memory, of which 42 MByte are in RSS (real memory). There is not much running on the machine apart from Slimserver 6.2 and it has 256 physical RAM and 300 MByte swap (which is mostly unused, i.e. the machine does not page).

I am not saying that exactly version 6.2 does make the big difference, but I have the impression that older versions (6.0 was my first) were faster indeed.

docbee
2005-10-25, 23:29
@jlynch3: the box delivers other web content at no delay. Just slimserver is behaving that slow.

@mherger: no plugins like moodlogic, musicmagic, etc. How many titles do you have in your archive? Maybe you are on the lucky side because of a smaller database?

@kdf: sorry I don't want to be unfair, but from my perspektiv the response time now is indeed very poor. I just speak for my machine.

Does someone have a crisp interface with a databse with more than 30.000 titles? Maybe there is a number of titles threshold where slimserver starts degrading in performance. Again, just guessing.

@meyergru: Interesting, that at least gives some indication that the database runs wild.

Is there an option to tell slimserver not to use sqllite but to hold everything in memory? If so, I would like to test this.

nmizel
2005-10-26, 01:42
For my part I noticed a slight overall speed increase compared to 6.1, except when the server refreshes the home page and re-count the number of albums / songs / artists.

I then occasionally get dropouts (ok, I have a slow hardware, but still...). If I remember right, this information used to be in the metainformation in 6.1.

For the time being, I'll hack the Default template and remove this information (which isn't that important for me) to get rid of this problem.

Cheers,
Nicolas

mherger
2005-10-26, 02:19
> @mherger: no plugins like moodlogic, musicmagic, etc. How many titles
> do you have in your archive? Maybe you are on the lucky side because of
> a smaller database?

It is smaller: about 6500.

> Is there an option to tell slimserver not to use sqllite but to hold
> everything in memory? If so, I would like to test this.

There's some information available on wiki.slimdevices.com on how to use
MySQL instead. MySQL knows in memory tables...

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
SlimString Translation Helper (http://www.herger.net/slim/)

meyergru
2005-10-26, 05:34
Aha. The display of the number of songs on the start page explains the read accesses.

I dug into the subject a little further and tried DBD::RAM (which is outdated and does not work any more) and DBD::AnyData instead of SQLite. To my dismay, I found that AnyData speaks a slightly different SQL dialect, so that the dbcreate.sql would have to undergo heavy changes in order to make that work.

However, I found that SQLite is one mighty tool that has not yet been used to its full potential by Slimserver.

As you can see here: http://www.sqlite.org/pragma.html, there is room for improvement.

When you insert the following code:

$dbh->do("PRAGMA temp_store = MEMORY");
$dbh->do("PRAGMA cache_size = 60000");

into .../Slim/DataStores/DBI/DataModel.pm at line 184 (just before the "$dbh->commit()"), you get rid of the temp-file creation and you can cache all (or at least most) of the database in memory. This comes at a cost of ~90 MBytes maximum increased memory consumption using my example of 60000 for cache_size (this is according to the SQLite specs). Matter-of-fact, my memory consumption went from 45M to 72M, which is neglegible.

One could use other PRAGMAs, like "synchronous = OFF", but I have not made it work, maybe because Slimserver uses AutoCommit=0.
Also, you can use "default_cache_size = XXX", this should make the cache size persistent in the database, so that further Slimserver updates do not neccessarily need the second improvement any more if you keep the database.

For me, these modifications make it really fast and what's more: the temp files are avoided completely (this obviously has wider implications than performance by itself).

docbee
2005-10-26, 07:37
That is good news. I will try this. Too good to be true, or to take it the other way round: argghhh!

docbee
2005-10-26, 08:05
I gave it a try and the application now behaves much crisper :-) What still bothers me is the long delay it takes to connect to the slimserver homepage for the first time.

I can hardly believe that slimserver does count all the titles, albums, etc via sql and is not storing these values. The only way these values could change is by doing a rescan, don't they? :-(

Ben Sandee
2005-10-26, 08:17
On 10/26/05, docbee <docbee.1xilun (AT) no-mx (DOT) forums.slimdevices.com> wrote:
>
>
> I gave it a try and the application now behaves much crisper :-) What
> still bothers me is the long delay it takes to connect to the
> slimserver homepage for the first time.
>
> I can hardly believe that slimserver does count all the titles, albums,
> etc via sql and is not storing these values. The only way these values
> could change is by doing a rescan, don't they? :-(


FWIW, I made the same change on my system and didn't see any improvement at
all (not that I was complaining about performance in the first place -- I
wasn't). So YMMV. :-)

Debian Sarge, Kernel 2.6.11, Athlon 800mhz, 512mb RAM, 1000 albums MP3

Ben

docbee
2005-10-26, 08:29
Ben, if you don't had an issue before it is no surprise to me that you don't see an improvement. Just put another 2000 albums on top an join the club...

meyergru
2005-10-26, 08:42
FWIW, I made the same change on my system and didn't see any improvement at
all (not that I was complaining about performance in the first place -- I
wasn't). So YMMV. :-)

Debian Sarge, Kernel 2.6.11, Athlon 800mhz, 512mb RAM, 1000 albums MP3



Sure. 1000 albums means only around 15000 songs, so you should not expect much difference.



I can hardly believe that slimserver does count all the titles, albums, etc via sql and is not storing these values. The only way these values could change is by doing a rescan, don't they? :-(


Maybe the difference between now and then really is the statistics info on the start page. Alas, the database table named "metainformation" cannot store the number of albums and artists, since there are only three columns:

version, track_count and total_time

So I guess this explains the full table scans for the stats...

And from what I have learned from the source, there is only a generic parameterized function (find) to do any database find, including counting. That function does even discriminate between specific database features of MySQL and SQLite, BTW...

Dan Sully
2005-10-26, 09:40
* meyergru shaped the electrons to say...

>So I guess this explains the full table scans for the stats...

We need to do a full scan with joins to get the proper counts - as now things
like including composer, etc and various artist albums are dynamic.

People seem to like to complain that it's both slow, and their counts are off
(from, say iTunes). So, which is the lesser of evils? If the counts are off,
"not all my music is being found!".

Good work on tracking down the PRAGMA issues though - I'll have a look.

-D
--
( ( ( [ ] ) ) )
In Stereo Where
Available

docbee
2005-10-26, 10:03
Dan, I don't use iTunes, maybe that's the reason why I don't understand. Do you really mean that the number of albums and artists does change in the database without doing a rescan? I had the impression that the only way to get things in/out is by initiating a rescan and therefore the stats computing should clearly be related to this function. But may be I didn't get it...

meyergru
2005-10-26, 10:09
We need to do a full scan with joins to get the proper counts - as now things like including composer, etc and various artist albums are dynamic.


They are dynamic? Why? I thought that there were very limited occasions on where modfiying accesses are needed for a music database, e.g. media scans or manipulating a playlist (even then, only the playlist can be changed, not the database itself). Normally, the database should be mostly static, or am I wrong?

You mentioned iTunes, does this add any demands in this respect that I missed?

Edit: Oops. docbee was faster than me...

docbee
2005-10-26, 10:45
Just for testing purpose I changed the following lines in Pages.pm to get rid of the stats sql traffic each time the startpage is displayed.

lines 188-190 changed to:
$params->{'song_count'} = 'any';
$params->{'artist_count'} = 'any';
$params->{'album_count'} = 'any';

and voila, now the startpage comes without waiting for the next coffee break ;-), which is extremly handy when working with the web interface.

From my perspective this stats information is never worth the performance drawback, but I still not convinced that you can't get both.

Dan Sully
2005-10-26, 10:56
* docbee shaped the electrons to say...

>Dan, I don't use iTunes, maybe that's the reason why I don't understand.
>Do you really mean that the number of albums and artists does change in
>the database without doing a rescan? I had the impression that the only
>way to get things in/out is by initiating a rescan and therefore the
>stats computing should clearly be related to this function. But may be
>I didn't get it...

I was using iTunes as an example of what people compare album counts, etc to.

The data doesn't change without a rescan / adding new music via Browse Music Folder.

However - we have dynamic views on that data. If a user doesn't want to
display any Composers in their artist list, that's a setting that can be
changed. The Artist count number will now need to go down.

Same thing for the new Various Artists albums - that's all dynamic against the database.

-D
--
<dr.pox> what're the units of the coefficient of agnosticity? I don't knows per hour?

meyergru
2005-10-26, 11:12
The data doesn't change without a rescan / adding new music via Browse Music Folder.

However - we have dynamic views on that data. If a user doesn't want to display any Composers in their artist list, that's a setting that can be changed. The Artist count number will now need to go down.

Same thing for the new Various Artists albums - that's all dynamic against the database.


BUT: All this is clearly defined. You need to recalculate the counts (with its 10 second impact) only on two occasions:

1. After scanning the music archive.
2. When changing settings (at least "various artists" and "contributor" settings as you wrote).

But not on:

3. Every time the start page is shown.

If you add columns for songs_count and album_count to the table "metainformation" and write those entries only when needed, you can simply retrieve that (static) info in Pages.pm when you build the start HTML page.

Alternatively and even easier, you can leave the redesign of the database out, use global static variables for the purpose and add another trigger for the full table scan:

0. At the start of the program (which may already be done during an initial rescan of the archive, so it is included in 1.).



Voila. Several seconds saved on every web page access.

ModelCitizen
2005-10-26, 11:57
Voila. Several seconds saved on every web page access.
SlimDevices... give this bloke a job... but I guess he probably doesn't need one.
MC

mikerob
2005-10-26, 15:23
Just to echo others' comments, I've seen an massive slow-down in browser response with 6.2.

It is taking 10-15 seconds to navigate between browser pages and the search function is taking 30-40 seconds to return results.

I'm running OSX 10.4.2 on a 1.42 GHz Mac Mini / 1G RAM - the Mac is only used for Slimserver - I've got 400 albums / 4400 tracks so not a large collection compared to some.

I run Music Magic Mixer but don't use iTunes or other third party plug-ins.

Both Safari and Firefox seem equally slow. I'm using the Default skin.

The Mac CPU doesn't seem to be maxing out - when doing a search or moving between browser page the perl process seems to tick away at around 10% CPU

I ran a lot of the 6.2 nightlies and don't remember the browser response being this slow before - in particular seach used to be really quick.

What debug information would be useful to find out what is going on?

meyergru
2005-10-26, 16:24
SlimDevices... give this bloke a job... but I guess he probably doesn't need one.


You bet! Anyway, thanks for the flattering recommendation.



What debug information would be useful to find out what is going on?

No debug info needed any more. We already know what is going on.

There is a full database scan (which is very expensive performance-wise) done in 6.2 in order to display the statistics information on the start page (i.e. # of songs, albums, artists). This operation seems to be done on any web page delivery, just for populating the needed parameters.

I have shown a way to avoid this and still display the information, I hope Dan Sully will chose to employ that scheme in a follow-up version.

A temporary fix would be to edit %SLIMSERVER%/Slim/Web/Pages.pm at line 188-190 where it says:

$params->{'song_count'} = _lcPlural($ds->count('track', $find), 'SONG', 'SONGS');
$params->{'artist_count'} = _lcPlural($ds->count('contributor', $find), 'ARTIST', 'ARTISTS');
$params->{'album_count'} = _lcPlural($ds->count('album', $find), 'ALBUM', 'ALBUMS');

Change those three lines to read:

$params->{'song_count'} = 0;
$params->{'artist_count'} = 0;
$params->{'album_count'} = 0;

By doing this, you lose the statistics on the start page but web access is just as fast as before 6.2.

The reason you experience this with a relatively small number of songs may be that your machine is not too fast, especially the harddisk (quote: "Since hard disk performance is the Mini's worst attribute, it makes sense for Apple to offer a faster disk for a bit more money, as an option. It may even be more sensible to make a faster drive the default in the 1.42 GHz model.").

A little further up in the thread, I have also shown a tuning tip for SQLite using PRAGMAs to speed up Slimserver database operations even more by sacrificing some memory (of which you have plenty) for a larger cache.

mikerob
2005-10-27, 02:44
The reason you experience this with a relatively small number of songs may be that your machine is not too fast, especially the harddisk (quote: "Since hard disk performance is the Mini's worst attribute, it makes sense for Apple to offer a faster disk for a bit more money, as an option. It may even be more sensible to make a faster drive the default in the 1.42 GHz model.").



Thanks for the interesting insight into what is going on - I'll try the hack later today.

I know the Mac Mini isn't the fastest machine on the market but personally I think any properly spec'd Mac less than 12 months old and still commercially available should be able to run Slimserver with ease! The drive is perfectly adequate for all but the most demanding applications - and Slimserver 6.2 seems to have become one of them...

I totally agree that I only see the justification to check the music library stats when you do something to change them. With me, that's only when I do a rescan as I don't use iTunes, composer, various artists or compilation album features.

meyergru
2005-10-27, 08:32
I totally agree that I only see the justification to check the music library stats when you do something to change them. With me, that's only when I do a rescan as I don't use iTunes, composer, various artists or compilation album features.

No, that is not the only opportunity:

It is neither those "features" nor iTunes but some new user preferences that make the difference here.

For example, depending on what ID3 fields you choose to be an "artist" (e.g. "componist", "band/orchestra" or "dirigist") in addition to the normal ID3 "artist", the number of distinct "artists" changes. With 6.2, this was implemented by a dynamic approach for calculating that number dependent on the user prefs.

With albums, there is a similar new logic when there are samplers. Since the ID3 artist can be different for each song in a sampler, the albums seem to be separated according to being located in the same subdirectory. A plain name comparison would not help, since there may also be albums from separate artists with the same name. The behaviour here can be changed to the old one (neglecting those subtle differences) by a user preference, too.

So, as you can see, there can be reasons apart from a rescan to change the counts.

My suggestion was to recalculate the counts only in the event of a rescan or if the preferences change.

mikerob
2005-10-27, 12:19
A temporary fix would be to edit %SLIMSERVER%/Slim/Web/Pages.pm at line 188-190 where it says:

$params->{'song_count'} = _lcPlural($ds->count('track', $find), 'SONG', 'SONGS');
$params->{'artist_count'} = _lcPlural($ds->count('contributor', $find), 'ARTIST', 'ARTISTS');
$params->{'album_count'} = _lcPlural($ds->count('album', $find), 'ALBUM', 'ALBUMS');

Change those three lines to read:

$params->{'song_count'} = 0;
$params->{'artist_count'} = 0;
$params->{'album_count'} = 0;

By doing this, you lose the statistics on the start page but web access is just as fast as before 6.2.



I tried this and the browser navigation is virtually immediate compared to 10-15 seconds previously. A great improvement over the original 6.2. Would be great to get this speed + the album/artist/track counts...

nmizel
2005-10-27, 13:14
I opened bug 2385 http://bugs.slimdevices.com/show_bug.cgi?id=2385 for this problem.

hashref
2005-10-27, 13:23
Ok...lines 188-190 are now as follows:

$params->{'song_count'} = 0;
$params->{'artist_count'} = 0;
$params->{'album_count'} = 0;

The startpage loads much quicker. Why do I still have my Album, Song, and Artist counts populated on the start page? I was going to add my own code to store this info in a cookie set to expire when the browser is closed but now that I still have these counts I'm a little confused.... I've tried from three different pc's, one which has never accessed my SlimServer. Is this info being cached on the server? I havent restarted SlimServer or the server itself.

meyergru
2005-10-27, 13:51
O.K. So there.

Apply this diff to .../Slim/Web/Pages.pm:



*** Slim/Web/Pages.pm.orig Mon Oct 24 23:47:21 2005
--- Slim/Web/Pages.pm Thu Oct 27 22:15:26 2005
***************
*** 157,162 ****
--- 157,171 ----
return sprintf("%s %s", $count, $word);
}

+ my $song_count = 0;
+ my $artist_count = 0;
+ my $album_count = 0;
+ my $genre_count = 0;
+
+ sub resetStats {
+ $song_count = 0;
+ }
+
sub addLibraryStats {
my ($params, $genre, $artist, $album) = @_;

***************
*** 185,200 ****
$find->{'contributor.role'} = $ds->artistOnlyRoles;
}

! $params->{'song_count'} = _lcPlural($ds->count('track', $find), 'SONG', 'SONGS');
! $params->{'artist_count'} = _lcPlural($ds->count('contributor', $find), 'ARTIST', 'ARTISTS');
! $params->{'album_count'} = _lcPlural($ds->count('album', $find), 'ALBUM', 'ALBUMS');

# Right now hitlist.html is the only page that uses genre_count -
# which can be expensive. Only generate it if we need to.
if ($params->{'path'} =~ /hitlist/) {

! $params->{'genre_count'} = _lcPlural($ds->count('genre', $find), 'GENRE', 'GENRES');
}
}

# Send the status page (what we're currently playing, contents of the playlist)
--- 194,216 ----
$find->{'contributor.role'} = $ds->artistOnlyRoles;
}

!
! if ($song_count eq 0) {
! $song_count = _lcPlural($ds->count('track', $find), 'SONG', 'SONGS');
! $artist_count = _lcPlural($ds->count('contributor', $find), 'ARTIST', 'ARTISTS');
! $album_count = _lcPlural($ds->count('album', $find), 'ALBUM', 'ALBUMS');
! }

# Right now hitlist.html is the only page that uses genre_count -
# which can be expensive. Only generate it if we need to.
if ($params->{'path'} =~ /hitlist/) {

! $genre_count = _lcPlural($ds->count('genre', $find), 'GENRE', 'GENRES');
}
+ $params->{'song_count'} = $song_count;
+ $params->{'artist_count'} = $artist_count;
+ $params->{'album_count'} = $album_count;
+ $params->{'genre_count'} = $genre_count;
}

# Send the status page (what we're currently playing, contents of the playlist)



And this to .../Slim/Music/Import.pm:



*** Slim/Music/Import.pm.orig Mon Oct 24 23:47:21 2005
--- Slim/Music/Import.pm Thu Oct 27 22:14:39 2005
***************
*** 7,12 ****
--- 7,13 ----

use strict;

+ use Slim::Web::Pages;
use Slim::Music::Info;
use Slim::Utils::Misc;
use Slim::Utils::Strings qw(string);
***************
*** 178,183 ****
--- 179,186 ----

$ds->cleanupStaleEntries();
}
+
+ Slim::Web::Pages->resetStats();
}
}




This does the database scan once at start and after each rescan (although I did not test that). It does not change the counts when you change the preferences, though. The logic employed in that area was beyond my comprehension and is left as an exercise to the reader ;-)

This quick hack works for now, but is a mess architecture-wise, so I would object to putting that into the code as a permanent fix.

mherger
2005-10-27, 14:24
> We need to do a full scan with joins to get the proper counts - as now
> things
> like including composer, etc and various artist albums are dynamic.

There's an optimisation in DBIStore.pm around page 325:
# Optimize the all case
if (scalar(keys %findCriteria) == 0) {

I don't know how much this would speed up the count. But it's rarely used
as Pages.pm adds the contributor.role to %findCriteria even if it's
undefined. The following patch would allow for the optimisation.

Index: D:/eclipse/SVN/Slim/Web/Pages.pm
================================================== =================
--- D:/eclipse/SVN/Slim/Web/Pages.pm (revision 4873)
+++ D:/eclipse/SVN/Slim/Web/Pages.pm (working copy)
@@ -180,7 +180,7 @@

$find->{'album.compilation'} = 1;

- } else {
+ } elsif (defined($ds->artistOnlyRoles)) {

$find->{'contributor.role'} = $ds->artistOnlyRoles;
}

There's little impact with my test notebook's 500 songs, though...

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

meyergru
2005-10-27, 15:04
>
There's an optimisation in DBIStore.pm around page 325:
# Optimize the all case
if (scalar(keys %findCriteria) == 0) {

I don't know how much this would speed up the count. But it's rarely used
as Pages.pm adds the contributor.role to %findCriteria even if it's
undefined.

...

There's little impact with my test notebook's 500 songs, though...


That might save the full table scan for calculating song_count, but the scans for album_count and artist_count are still being done, so maybe this is why you don't see much improvement from that correction (I have not looked into this since the scan is done only once using my hack anyway).

The file DBIStore.pm seems to be the right place to store the global values, since that is where this optimization is done (as you found, incorrectly) for song_count (aka trackCount). They need not even be in the metainformation table, a global variable as in my hack should suffice.

Whatever, that's all up to Dan Sully... I'm out now.

mherger
2005-10-27, 21:14
>> There's little impact with my test notebook's 500 songs, though...
>
> That might save the full table scan for calculating song_count, but the
> scans for album_count and artist_count are still being done, so may this

If I remember right it saved the full scan for all but artist_count. (2
out of 3)

> is why you don't see much improvement from that correction.

The reason is rather the fact that with that little number of songs
there's no noticable delay anyway.

> Whatever, that's all up to Dan Sully... I'm out now.

Right. Let's see what he means about all this.

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

Dan Sully
2005-10-28, 12:52
* meyergru shaped the electrons to say...

>That might save the full table scan for calculating song_count, but the
>scans for album_count and artist_count are still being done, so may this
>is why you don't see much improvement from that correction.

Can someone with this slow down issue please send me their slimserversql.db file?

dan | at | slimdevices.com

Thanks.

-D
--
Ya gotta love UNIX, where else do you wonder whether
you can kill a zombie spawned by a daemon's fork?

mikerob
2005-10-28, 14:15
slimserversql.db file emailed to you Dan

nmizel
2005-10-28, 14:23
Hello Dan,

MySQL 4.1.12 dump file available at ftp://sirius.mizel.net/pub/slimserver.dump (anonymous login)

SlimServer Version: 6.2.0 - 4753 - Linux - EN - utf8

Nicolas

MrC
2005-10-28, 16:29
Dan,

I just tried running some tests with your web page timer code. There is something else going on, that is not being included in your times.

Most pages took anywhere from .5 to 3 seconds to load in my test. Then, I closed the page in my browser, and re-opened a page to the server. It took nearly 9 seconds to render, yet the output of the page render times showed 2 seconds wall clock time, and 2.05 CPU time. Clearly there are about 7 seconds not accounted for in wallclock time.

Dan Sully
2005-10-28, 16:33
* MrC shaped the electrons to say...

>I just tried running some tests with your web page timer code. There
>is something else going on, that is not being included in your times.
>
>Most pages took anywhere from .5 to 3 seconds to load in my test.
>Then, I closed the page in my browser, and re-opened a page to the
>server. It took nearly 9 seconds to render, yet the output of the page
>render times showed 2 seconds wall clock time, and 2.05 CPU time.
>Clearly there are about 7 seconds not accounted for in wallclock time.

Keep in mind that the time displayed is from immediately before generating
the page to immediately after. It doesn't take into account transfer time
afterwards to send the page to your browser.

-D
--
<iNoah> you know, most free operating systems come preinstalled with their own high horse.

MrC
2005-10-28, 16:43
Keep in mind that the time displayed is from immediately before generating
the page to immediately after. It doesn't take into account transfer time
afterwards to send the page to your browser.

Yeah, understood, and that's what I suspected too. So, the question is, what's going on underneath the covers that makes this thing hicup every now and again? There are long, unpredicatable stalls.

I just tested again, and this time opening the page took 15 seconds. No data is coming from the server to the browser during most of this time.

Anyway, I'm not meaning to push for a response here and now; rather, i'm just trying to add some data points to this general performance discussion. I think similar perfmon calls surrounding other core areas of code would be fantastic.

dean
2005-10-28, 16:48
Does the skin make a difference? How are the load times on the EN
(Lite) skin?


On Oct 28, 2005, at 4:43 PM, MrC wrote:

>
> Dan Sully Wrote:
>
>> Keep in mind that the time displayed is from immediately before
>> generating
>> the page to immediately after. It doesn't take into account transfer
>> time
>> afterwards to send the page to your browser.
>>
>>
> Yeah, understood, and that's what I suspected too. So, the question
> is, what's going on underneath the covers that makes this thing hicup
> every now and again? There are long, unpredicatable stalls.
>
> I just tested again, and this time opening the page took 15 seconds.
> No data is coming from the server to the browser during most of this
> time.
>
> Anyway, I'm not meaning to push for a response here and now; rather,
> i'm just trying to add some data points to this general performance
> discussion. I think similar perfmon calls surrounding other core
> areas
> of code would be fantastic.
>
>
> --
> MrC
>

Dan Sully
2005-10-28, 16:48
* MrC shaped the electrons to say...

>Yeah, understood, and that's what I suspected too. So, the question
>is, what's going on underneath the covers that makes this thing hicup
>every now and again? There are long, unpredicatable stalls.

That is an interesting question.. :)

>Anyway, I'm not meaning to push for a response here and now; rather,
>i'm just trying to add some data points to this general performance
>discussion. I think similar perfmon calls surrounding other core areas
>of code would be fantastic.

Patches welcome.

-D
--
<dr.pox> NO, NETBSD IS NOT REALLY BUILT WITH ELITE FORTRAN77!!@$#$

MrC
2005-10-28, 17:56
Does the skin make a difference? How are the load times on the EN
(Lite) skin?

Yes, skin seems be a factor, as does browser.

Lite is very fast, never stalls.

ExBrowse3 seems fine too.

Fishbone (sorry KDF) seems to stall as much as 18 seconds on Opera and Firefox. Never on IE.

I'm running the latest svn from Trunk.

kdf
2005-10-28, 18:24
On 28-Oct-05, at 5:53 PM, MrC wrote:
>
> Load times are consistently lightning fast with Lite. I can generate
> the stalls frequently (sorry KDF) with Fishbone. ExBrowse3 seems
> fine.
>
the price you pay for my bad code and my desire to have all those
commands one click away ;) Once I can make sense of all of Jacobs
work, I'm hoping to find the time to make a fishbone2 using the ajax
tools (and maybe skin inheritance once implemented in 6.5)

-k

MrC
2005-10-28, 18:27
Its a fair price. How can anyone look a gift horse in the mouth!

Dan Sully
2005-10-28, 19:07
* Mark Lanctot shaped the electrons to say...

>1. Nearly every time I connect to the Squeezenetwork,
>it asks me to do a firmware update. I have updated
>the firmware at least 20 times tonight! The splash
>screen on startup keeps alternating between
>"squeezebox" (i.e. the new one) and "squeezebox2". I
>have a feeling it's going from the new firmware back
>to the old one again and again. Any advice to get out
>of this loop? I understand that the newest SlimServer
>also contains the latest firmware, but I'm running
>6.1.1 - is this part of the problem?

I just rolled out a new version of SqueezeNetwork - it should stop doing this
as of ~20 minutes ago.

-D
--
"You can usually recover from production flaws...but you can never recover from a bad design".

Jacob Potter
2005-10-28, 19:41
On 10/28/05, kdf <slim-mail (AT) deane-freeman (DOT) com> wrote:
> Once I can make sense of all of Jacobs
> work, I'm hoping to find the time to make a fishbone2 using the ajax
> tools (and maybe skin inheritance once implemented in 6.5)

I'm definitely waiting for the skin inheritance feature before putting
up Default3; I'd do it myself if I was more adept at Perl. Ideally,
with inheritance and the new command wrappers, the only thing that
would need rewriting is index.html and the CSS.

(I think this is the first time I've heard of one of my skins being
*faster* to load than a non-scripted one... :) )

- Jacob

meyergru
2005-10-29, 05:07
Me again.

The fact is that with large databases, a massive improvement can be seen from avoiding the scans in Pages.pm.

As far as I can tell that scan is being done for any skin whatsoever, even if the statistics info is not being used after it was generated.

If some people experience skin- or browser-related performance issues, I would guess they be caused by something different, so we should be careful not to mix up those two matters here.

I have not seen hangs from HTML delivery with certain browsers or skins yet, but if that is the case, I would point to something like HTTP version differences. HTTP/1.1 is able to do pipelining, BTW. There are settings in Firefox and Mozilla to influence that behaviour, so one could try that.

Also, there are PERL tricks for buffering HTTP headers vs. data correctly, as is explained here: http://httpd.apache.org/docs/1.3/misc/FAQ.html#premature-script-headers

Normally, this should be handled transparently by standard libraries like CGI.pm, but I don't know if the Slimserver code in that area is self-made or reused.

mherger
2006-02-02, 15:16
A temporary fix would be to edit %SLIMSERVER%/Slim/Web/Pages.pm at line 188-190 where it says:

$params->{'song_count'} = _lcPlural($ds->count('track', $find), 'SONG', 'SONGS');

Change those three lines to read:

$params->{'song_count'} = 0;


This temporary fix has been made an official option in the latest nightly.

JJZolx
2006-02-02, 15:30
I like the idea of being able to suppress the stats display. Who in the world needs to be reminded of this on every single page? Put it on a single page in the server setup and then nobody will really care how slow it is.

Michaelwagner
2006-02-02, 17:20
This temporary fix has been made an official option in the latest nightly.
6.2.2 or 6.5?

mherger
2006-02-02, 22:30
>> This temporary fix has been made an official option in the latest
>> nightly.
> 6.2.2 or 6.5?

Both.

--

Michael

-----------------------------------------------------------
Help translate SlimServer by using the
StringEditor Plugin (http://www.herger.net/slim/)

oreillymj
2006-02-03, 03:06
Just downloaded the nightly with this change and the UI is so much more responsive it incredible. Re-scans also seem to be faster.

Still had to downgrade to FW28, as 29 doesn't work with my router.

lx in space
2006-02-09, 15:31
slimserver has been messing with my head

however tonight i tried firefox rather than safari, and huge performance improvements (on both 6.2.1 and 6.2.2 - have just gone back to 6.2.1 as seemed to be getting some choppy audio with the 09/02 build)

no idea why of course!

for ref, mac mini, 512MB ram, library on external fw drive, around 6000 songs, am using mySQL, rather than SQLite

lx