PDA

View Full Version : Scans seem slow? web, SBC ui slow? how to make them faster! vote for this bug!



MrSinatra
2009-10-25, 11:56
if you feel that scans take too long, the UI's are too slow, read this thread:

http://forums.slimdevices.com/showthread.php?t=60682

and vote for this bug:

http://bugs.slimdevices.com/show_bug.cgi?id=14921

hopefully, eventually the optimization will be automated at install. in the meantime, there are the instructions, but BEWARE, this is AT YOUR OWN RISK!!! make copies FIRST of everything you change!!! don't do this unless you feel comfortable trying it, but if you do you should notice substantial improvements.


a "win-xp-dummies" guide to installing moonbases tweaks, but easily translated to most windows/slimware.

make sure you know your last scans stats prior to any of this!

1. download moonbase's my.tt.txt from post 18

http://forums.slimdevices.com/showpost.php?p=401530&postcount=18

2. stop SBS and exit all SBS functions. (check task manager to be sure all squeeze stuff, slim stuff, mysql/mysqld, etc... is gone/closed)

3. find the SBServer cache folder, which is normally located at:

C:\Documents and Settings\All Users\Application Data\Squeezebox\Cache

and delete a file called my.cnf which might appear as a shortcut w/an icon of a monitor simply named: my

4. backup your current copy of my.tt which is normally located at:

C:\Program Files\Squeezebox\server\MySQL

to a safe place. then delete it from its original location.

5. copy moonbases downloaded file to the my.tt original location.

6. rename moonbase's file to remove the .txt at the end.

7. if desired, reboot the computer (safer). if not, simply restart SBS.

8. run a full "clear and rescan" and see how much better it is, then report both results back here to this thread.

moonbase's file assumes you have at least a gig of ram and other reasonably new hardware. part of what we want to know is what hardware it works well with, and what hardware it doesn't.

verypsb
2009-10-26, 09:49
Is this not too much work, when there soon will be a switch to SQLite from MySQL?

signor_rossi
2009-10-26, 10:12
Is this not too much work, when there soon will be a switch to SQLite from MySQL?

What is soon? :)
The release of the Touch?

MrSinatra
2009-10-26, 12:35
i'm not 100% positive, but i believe there will be a variation of this applicable to the forthcoming SQLite ver of SBS. AndyG needs our data in order to implement the feature.

verypsb
2009-10-26, 13:05
System:
Windows 2008 R2 (x64), 4GB, Pentium Dual-Core E5200@2.50Ghz, 99% .flac, OS on 1 single drive, music files on 2 drives in RAID0 (Intel ICH8R/ICH9R/ICH10R/DO SATA RAID Controller)

Total Tracks: 35,792
Total Albums: 3,445
Total Artists: 1,033
Total Genres: 151
Total Playing Time: 2767:39:29

Music Scan Details (Before)
Directory Scan (35792 of 35792) Complete 00:35:37
Playlist Scan ( of ) Complete 00:00:00
MusicIP Import (35321 of 35321) Complete 00:24:03
Merge Various Artists (3309 of 3309) Complete 00:01:27
Artwork Scan (3445 of 3445) Complete 00:24:36
The server has finished scanning your music collection.
Total Time: 01:25:43 (Monday, 26 October 2009 / 19:24)

Music Scan Details (After)
Directory Scan (35792 of 35792) Complete 00:33:56
Playlist Scan ( of ) Complete 00:00:00
MusicIP Import (35321 of 35321) Complete 00:13:57
Merge Various Artists (3309 of 3309) Complete 00:01:26
Artwork Scan (3445 of 3445) Complete 00:24:34
The server has finished scanning your music collection.
Total Time: 01:13:53 (Monday, 26 October 2009 / 20:56)

MrSinatra
2009-10-26, 13:12
interesting... your directory scan part only improved a min or two, but your musicip part improved by around 10min. in my case, i don't use musicip, but my directory scan improved by about 9min. i think other people will get better results since my music is on an ext usb2 drive.

hard drive cranking, any issues like that? webui response better?

verypsb
2009-10-26, 23:41
webui response better?

Yes, the WebUI looks more responsive.

raven22
2009-10-27, 00:28
where can i find the my.tt file in a debian install?


edit: found it! it is in /usr/share/squeezeboxserver/MySQL

MrSinatra
2009-10-27, 00:31
where can i find the my.tt file in a debian install?

i don't know, isn't it roughly in the same spot? did u try reading the whole thread at the first link i posted?

mherger
2009-10-27, 02:04
> where can i find the my.tt file in a debian install?

On Linux systems the pre-installed database server is being used. The changes discussed would need to be applied to the main mysql instance. Configuration will influence other apps using it too. Be sure what you're doing.

Configuration files most likely are in /etc/my.cnf or similar.

MrSinatra
2009-10-27, 02:08
where can i find the my.tt file in a debian install?


edit: found it! it is in /usr/share/squeezeboxserver/MySQL

just in case you missed his edit.

raven22
2009-10-27, 03:30
> where can i find the my.tt file in a debian install?

On Linux systems the pre-installed database server is being used. The changes discussed would need to be applied to the main mysql instance. Configuration will influence other apps using it too. Be sure what you're doing.

Configuration files most likely are in /etc/my.cnf or similar.

Well, I changed the files according instructions on my Sheevaplug running Debian. It has 'only' 512mb memory and not the suggested at least 1GB.

Speed increase is incredible on 16000 files with artwork files in each album folder. First it took around 1 hour 30 minutes, now only 1 hour and 2 minutes.

MrSinatra
2009-10-27, 11:17
thats what we're trying to figure out... the default my.tt is set to work with even the most low power hardware, (think NAS, etc) so moonbase setup a tweaked file to unchoke the DB, but his values for it are really just guesses. the danger is the tweaks could outpower someones hardware, but we don't yet have a handle on what thresholds are where.

ultimately, the best solution is for hardware tests to be run during SBS install so the values are dynamically set (as well as library size and artwork calculations), but unless there is a pgm out there that already "knows" how to do that well (to optimize sql for differing hardware), we have to give AndyG datapoints to set different my.tt's based on what installer finds.

andyg
2009-10-27, 11:23
I saw some mention of SQLite so I wanted to clear up some confusion. SQLite has no tunable parameters like this. It is faster during scanning (no IPC traffic to MySQL) but not as fast during complex browse queries where MySQL is more advanced.

The current plan is to include full support for both MySQL and SQLite, with MySQL the default on non-embedded platforms. There will be a pref to switch database backends if you want.

usch
2009-10-27, 12:58
That's good news. I am still figuring out how to migrate my iTunes smart playlists to MySQL playlists, and was a bit worried that these might not be compatible with the next major release.

jonstahl
2009-10-28, 21:46
I ran this on my 2Ghz (celeron) Win XP box, 512MB RAM, music collection on an external USB2 drive. (Yes, I know I'm a little light on RAM, but hopefully this helps you plumb the lower end.)

Before optimization:

Total Tracks: 23,880
Total Albums: 4,322
Total Artists: 2,164
Total Genres: 174
Total Playing Time: 1854:48:41

Music Scan Details
Directory Scan (23889 of 23889) Complete 01:05:39

Playlist Scan ( of ) Complete 00:03:31

MusicIP Import (23929 of 23929) Complete 00:26:29

Merge Various Artists (4228 of 4228) Complete 00:04:59

Artwork Scan (4321 of 4321) Complete 00:30:56

The server has finished scanning your music collection.
Total Time: 02:11:34


With optimized my.tt file:


Directory Scan (23889 of 23889) Complete 00:49:51

Playlist Scan (0 of ) Complete 00:03:30

MusicIP Import (23929 of 23929) Complete 00:21:42

Merge Various Artists (4227 of 4227) Complete 00:05:02

Artwork Scan (4321 of 4321) Complete 00:32:39

The server has finished scanning your music collection.
Total Time: 01:52:44

peterw
2009-10-28, 22:15
On Linux systems the pre-installed database server is being used. The changes discussed would need to be applied to the main mysql instance. Configuration will influence other apps using it too.

Not always. I'm running a SVN checkout of 7.3.x, and it's using a "my.cnf" file in the SqueezeCenter Cache directory. SqueezeCenter spawns separate mysqld processes for SqueezeCenter using mysqld binaries compiled by Logitech and pulled from Subversion. SVN checkouts of SBS 7.4 and SBS 7.5 behave the same.


Be sure what you're doing.

Yes, indeed. I suspect Michael's right about those using the RPM and .deb packages, but SVN behaves differently, and I expect the fat tarball releases may, too.

erland
2009-10-28, 22:45
Not always. I'm running a SVN checkout of 7.3.x, and it's using a "my.cnf" file in the SqueezeCenter Cache directory. SqueezeCenter spawns separate mysqld processes for SqueezeCenter using mysqld binaries compiled by Logitech and pulled from Subversion. SVN checkouts of SBS 7.4 and SBS 7.5 behave the same.

On the debian/ubuntu installation it uses the system installed mysqld but it still launches a separate instance of it with the Squeezebox Server specific my.cnf, it's started by Squeezebox Server like this:

/usr/sbin/mysqld --defaults-file=/var/lib/squeezeboxserver/cache/my.cnf

MrSinatra
2009-10-29, 02:39
erland, other guys, did you do the tweak? post the results! everyone so far has gotten some kind of boost from it...

radish
2009-10-29, 06:02
I'll do it tonight if I get a chance.

Wirrunna
2009-10-30, 15:46
Standard my.tt , system is XP SP3, Q9300, 4Gb RAM, music on 1Tb WD Green Power and 1.5Tb Samsung EcoGreen 154UI, both low performance 5400rpm disks.

Version: 7.3.4 - 28402 @ Sun Sep 13 03:01:44 PDT 2009
Operating system: Windows XP - EN - cp1252
Platform Architecture: 586
Perl Version: 5.8.8 - MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt
Total Players Recognized: 0

Music Scan Details
Directory Scan (97147 of 97147) Complete 01:29:11
MusicIP Import (97146 of 97146) Complete 01:01:52
Playlist Scan (21 of 21) Complete 00:00:19
Merge Various Artists (7417 of 7417) Complete 00:01:30
Artwork Scan (8107 of 8107) Complete 00:29:07
Database Optimize (0 of ) Complete 00:04:05

SqueezeCenter has finished scanning your music collection.
Total Time: 03:06:04 (Friday, October 30, 2009 / 1:50 PM)

Moonbase's modified my.tt
Music Scan Details
Directory Scan (97147 of 97147) Complete 01:14:23
MusicIP Import (97146 of 97146) Complete 00:30:50
Playlist Scan (21 of 21) Complete 00:00:13
Merge Various Artists (7417 of 7417) Complete 00:01:33
Artwork Scan (8108 of 8108) Complete 00:28:59
Database Optimize (0 of ) Complete 00:01:33

SqueezeCenter has finished scanning your music collection.
Total Time: 02:17:31 (Saturday, October 31, 2009 / 9:21 AM)

Significant speed up. I don't know why it found one more artwork, nothing was changed.
A bit of Googling on optimizing Mysql led me to this http://optimmysql.blogspot.com/2008/04/variable-day-out-4-innodbbufferpoolsize.html which has some good links for those who want to experiment further.

ModelCitizen
2009-10-31, 09:57
Summary:
Overall scan time decreased by around 11% (from 46 mins to 41 mins)
Note: Playlist scan time increased by a couple of seconds due to addition of new playlist between scans

PC Spec:
7.4.1 on XP Pro
2.5ghz. AMD Athlon Dual Core 4850e
3gb Ram
1tb Internal SataII music disk

Total Tracks: 25,258
Total Albums: 1,860
Total Artists: 858
Total Genres: 57
Total Playing Time: 2282:38:38

Pre-fix stats:
Directory Scan (25258 of 25258) Complete 00:23:22
Playlist Scan (20 of 20) Complete 00:00:09
MusicIP Import (24596 of 24596) Complete 00:11:58
Merge Various Artists (1721 of 1721) Complete 00:00:48
Artwork Scan (1860 of 1860) Complete 00:09:45
The server has finished scanning your music collection.
Total Time: 00:46:02 (Saturday 31 October 2009 / 12:07:57)

Post fix stats:
Directory Scan (25258 of 25258) Complete 00:21:06
Playlist Scan (21 of 21) Complete 00:00:11
MusicIP Import (24596 of 24596) Complete 00:09:08
Merge Various Artists (1721 of 1721) Complete 00:00:46
Artwork Scan (1860 of 1860) Complete 00:09:37
The server has finished scanning your music collection.
Total Time: 00:40:48 (Saturday 31 October 2009 / 16:11:01)

MC

Wirrunna
2009-10-31, 15:13
More experimantation, different server Q9450, music on 5400rpm WD Green Power 1Tb and Samsung 1.5Tb EcoGreen HD154UI, 4GbRAM. XP Pro SP3
Version: 7.3.4 - 28402 @ Sun Sep 13 03:01:44 PDT 2009
Standard my.tt
Directory Scan (97147 of 97147) Complete 01:12:22
MusicIP Import (97146 of 97146) Complete 00:46:40
Playlist Scan (21 of 21) Complete 00:00:14
Merge Various Artists (7730 of 7730) Complete 00:01:31
Artwork Scan (8431 of 8431) Complete 00:29:25
Database Optimize (0 of ) Complete 00:02:39

SqueezeCenter has finished scanning your music collection.
Total Time: 02:32:51 (Saturday, October 31, 2009 / 9:04 PM)

Installed Moonbase's my.tt and set innodb_buffer_pool_size = 500M.
Music Scan Details
Directory Scan (97147 of 97147) Complete 01:03:50
MusicIP Import (97146 of 97146) Complete 00:21:12
Playlist Scan (21 of 21) Complete 00:00:06
Merge Various Artists (7730 of 7730) Complete 00:01:27
Artwork Scan (8431 of 8431) Complete 00:28:43
Database Optimize (0 of ) Complete 00:00:31

SqueezeCenter has finished scanning your music collection.
Total Time: 01:55:49 (Sunday, November 1, 2009 / 12:17 AM)

During both scans I monitored memory usage of mysqld.exe with Windows Task Manager, standard my.tt used 18,242K, modified my.tt built up to a peak of 559,640K after scanning about 60,000 tracks. I also monitored disk i/o using Windows Process Monitor; with the standard my.tt mysqld.exe dominated the disk i/o, with the modified my.tt scanner.exe was the major disk i/o until 60,000 tracks then mysqld.exe occasionally did some disk i/o but only 3 or 4 Mb in a burst.

The web UI is now a pleasure to use. I had stopped using it since I bought iPeng as the ui was so slow and even caused re-buffering in the music, but it is now very fast for searches and to display album art.

After the scan, mysqld.exe was still using 580M of RAM, so I re-booted the server to see if it would release some RAM, which it does. It has now been running for an hour, I have done some music searches, browse albums and artists an the mysqld.exe is using only 146,808K, less than the 170,304K that SQUEEZ~1.EXE is using.

Conclusion - if you have plenty of RAM in your server, give the innodb_buffer_pool_size as much ram as you can, up to the size of C:\Documents and Settings\All Users\Application Data\SqueezeCenter\Cache\MySQL. mysqld.exe will use all it can on a clear and rescan but after a scan will only use what it needs.

aubuti
2009-11-01, 09:07
My system is an Atom single-core 1.6MHz (Intel 945GC) with 2GB RAM, although only 1.6GB is available because 0.4GB is allocated to a virtual machine, which was suspended while running these tests. The OS is Ubuntu 8.10 Desktop, and running SBS is about all the machine does.

The directory scan time is about 22% faster, although on my small library of 5132 tracks that's less than one minute. Other aspects of rescan are basically unchanged before and after the tweak. Total scan time is 11% faster, or almost exactly 1 minute out of 10:39.

Version: 7.4.1 - r28947 @ Tue Oct 20 07:58:02 PDT 2009
Hostname: pinot
Server IP Address: 192.168.0.21
Server HTTP Port Number: 9000
Operating system: Debian - EN - utf8
Platform Architecture: i686-linux
Perl Version: 5.10.0 - i486-linux-gnu-thread-multi
MySQL Version: 5.0.67-0ubuntu6
Total Players Recognized: 10

BEFORE TWEAK
Music Scan Details
Directory Scan (5132 of 5132) Complete 00:05:07
Playlist Scan (28 of 28) Complete 00:00:40
MusicIP Import (5131 of 5131) Complete 00:03:23
Merge Various Artists (397 of 397) Complete 00:00:25
Artwork Scan (403 of 403) Complete 00:01:04
The server has finished scanning your music collection.
Total Time: 00:10:39 (Sunday, November 1, 2009 / 10:40)

AFTER THE TWEAK
Music Scan Details
Directory Scan (5132 of 5132) Complete 00:04:11
Playlist Scan (28 of 28) Complete 00:00:40
MusicIP Import (5131 of 5131) Complete 00:03:22
Merge Various Artists (397 of 397) Complete 00:00:24
Artwork Scan (403 of 403) Complete 00:01:01
The server has finished scanning your music collection.
Total Time: 00:09:38 (Sunday, November 1, 2009 / 10:55)

EDIT: Will report back on web ui responsiveness. Earlier when I tried the my.tt tweak it seemed that for the slowest part of the web ui -- browsing albums with large album art and ~50 albums per screen -- that after the tweak it actually took *longer* to respond at first to a screen refresh, but then displayed the album art very quickly. So it seemed longer to go from 0 to 1, and faster to go from 1 to 50. I think the net result was basically no change, but I didn't measure it.

Random Lurker
2009-11-12, 09:22
Here are results on my massive library:

PC Spec:
Windows XP SP3
Pentium 4 - 3.0 Ghz
2.5 GB Ram
Most music on 2 external 1TB USB Western Digital Drives, some on internal 750 GB IDE drive.
Library size: 2.2 Terabytes, about 75% FLAC, the rest in MP3


Squeezebox Server Status
Version: 7.4.2 - r29203 @ Mon Nov 9 04:03:04 PST 2009
Operating system: Windows XP - EN - cp1252
Platform Architecture: 586
Perl Version: 5.10.0 - MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt
Total Players Recognized: 1

Music Scan Details
Directory Scan (147326 of 147326) Complete 03:18:19
Playlist Scan ( of ) Complete 00:00:00
Merge Various Artists (8235 of 8235) Complete 00:06:29
Artwork Scan (8244 of 8244) Complete 01:13:50

The server has finished scanning your music collection.
Total Time: 04:38:38 (Tuesday, November 10, 2009 / 2:09 AM)


-------------------------------------------------------------------
After adding moonbase's my.tt.txt (And cleaning up some ALBUMARTISTSORT issues):

Music Scan Details
Directory Scan (147326 of 147326) Complete 02:29:05
Playlist Scan ( of ) Complete 00:00:00
Merge Various Artists (8236 of 8236) Complete 00:05:47
Artwork Scan (8245 of 8245) Complete 01:10:42

The server has finished scanning your music collection.
Total Time: 03:45:34 (Wednesday, November 11, 2009 / 2:09 AM)


The total time dropped from 04:38:38 to 03:45:34 - a nice improvement. And more importantly, the responsiveness does have the promised gains. While the search function used to be worthless on my library due to how slow it was, it can now actually return results. Thanks for taking the time to dig into this.....

MrSinatra
2009-11-12, 10:26
this is great, but we need MORE results and MORE votes!!!

vote for the bug!!!

http://bugs.slimdevices.com/show_bug.cgi?id=14921

i'm surprised some of the old guard here hasn't posted their results yet. (even just an idea of how much improved, if you don't have pre tweak stats)

mherger
2009-11-12, 10:34
> this is great, but we need MORE results and MORE votes!!!

I eventually gave this a try, tweaking both the mysqld instance shipped with SBS as well as my system's configuration. None made a notable difference (1-2% at max). I haven't seen many Linux systems so far. Is this more important on Windows I wonder?

kdf
2009-11-12, 11:01
>> this is great, but we need MORE results and MORE votes!!!
>
> I eventually gave this a try, tweaking both the mysqld instance shipped
> with SBS as well as my system's configuration. None made a notable
> difference (1-2% at max). I haven't seen many Linux systems so far. Is
> this more important on Windows I wonder?


I use the system instance of mysql with my linux install. The changes are
irrelevant since the system file already matches the suggested value.

-kdf

MrSinatra
2009-11-12, 11:13
sheevaplug is linux right? made a big difference on that. in moonbases orig thread another linux user had a HUGE benefit, 4hr to 2.5 r u sure you implemented the tweak in the right place? linux seems to be more variable as to where the file should go. and how many files, etc?

Wirrunna
2009-11-12, 16:17
I have lived with a modded my.tt (see my posts earlier in this thread) with set innodb_buffer_pool_size set to 500m for two weeks, monitoring the memory usage via Windows Task Manager.
Unless I have performed a Clear & Rescan, mysqld.exe does not use more than about 125M. Lazy searches are extra fast, with the number of matches almost instant as I enter each character. The GUI is fast and as I said earlier a pleasure to use (this is with a library of 97,759 tracks, 8,473 albums, most with cover.jpg - mostly FLAC, but some mp3 and m4a).
Browsing by albums - Home > Albums - takes a few seconds going from one letter to another, but is still acceptable. Clicking on the artwork instantly brings up the album details.
Browsing by artists - Home > Artists - also takes a a couple of seconds between letters, but again clicking on an artist brings up an instant display of the albums.
Search Music - Home > Search -is nearly instant in displaying search results.
This mod is certainly worthwhile in Windows and Squeezecenter 7.3.4

radish
2009-11-12, 19:27
As promised, here are my results. Not a huge improvement in actual time, but it works out to around 10% which isn't terrible. This is on my "production" server which is an AthlonXP/2G RAM.

Version: 7.4.2 - r29220 @ Tue Nov 10 04:01:26 PST 2009
Hostname: gary
Server IP Address: 192.168.2.163
Server HTTP Port Number: 9000
Operating system: Debian - EN - utf8
Platform Architecture: i686-linux
Perl Version: 5.10.0 - i486-linux-gnu-thread-multi
MySQL Version: 5.0.75-0ubuntu10.2
Total Players Recognized: 9

==========
Before
==========

Directory Scan (21833 of 21833) Complete 00:21:45
Playlist Scan (8 of 8) Complete 00:00:06
Merge Various Artists (963 of 963) Complete 00:00:06
Artwork Scan (1580 of 1580) Complete 00:05:27
The server has finished scanning your music collection.
Total Time: 00:27:24

==========
After
==========

Directory Scan (21833 of 21833) Complete 00:19:23
Playlist Scan (8 of 8) Complete 00:00:04
Merge Various Artists (963 of 963) Complete 00:00:06
Artwork Scan (1580 of 1580) Complete 00:04:56
The server has finished scanning your music collection.
Total Time: 00:24:29

mherger
2009-11-13, 00:23
Ok, here go my numbers (this is a Via C3/1GHz, 1GB RAM box):

Version: 7.5.0 - rTRUNK @ UNKNOWN
BS: Red Hat - DE - utf8
Plattform: i686-linux
Perl-Version: 5.10.0 - i386-linux-thread-multi
MySQL-Version: 4.1.22

Before (SBS MySQL):
Durchsuche Musik-Verzeichnis (15435 von 15435) Beendet 00:32:34
Suche Wiedergabelisten (6 von 6) Beendet 00:00:01
MusicIP-Import (15481 von 15481) Beendet 00:18:07
Gruppiere "Diverse Artisten" (1156 von 1156) Beendet 00:02:25
Suche Plattenhüllen (1157 von 1157) Beendet 00:19:35
Der Server hat das Durchsuchen Ihrer Musiksammlung abgeschlossen.
Laufzeit: 01:12:42 (Freitag, 28. Oktober 2009 / 09:09)

After (using system's MySQL):
Durchsuche Musik-Verzeichnis (15503 von 15503) Beendet 00:27:59
Suche Wiedergabelisten (6 von 6) Beendet 00:00:00
MusicIP-Import (15481 von 15481) Beendet 00:16:35
Gruppiere "Diverse Artisten" (1156 von 1156) Beendet 00:02:14
Suche Plattenhüllen (1157 von 1157) Beendet 00:18:43
Der Server hat das Durchsuchen Ihrer Musiksammlung abgeschlossen.
Laufzeit: 01:05:31 (Freitag, 13. November 2009 / 00:29)


Hmm... that's actually more of an improvement than I thought I had seen the first time I tested this...

MrSinatra
2009-11-13, 03:46
guys, remember, scan times is just HALF the tweak. the other half is a dramatic increase in UI responsiveness.

also, we need prob 30 votes for this to get implemented officially, so everyone PLEASE VOTE!

http://bugs.slimdevices.com/show_bug.cgi?id=14921

aubuti
2009-11-13, 07:21
guys, remember, scan times is just HALF the tweak. the other half is a dramatic increase in UI responsiveness.
I admit I haven't done much systematic comparison of ui responsiveness, but the little I have done certainly didn't show dramatic differences. Maybe I'll do some more over the weekend. Any good recommendations for how best to *measure* ui responsiveness?

MrSinatra
2009-11-13, 07:33
some people have talked about doing searches or drilling down to artists or songs via different nav paths.

but if you mean a true benchmark, no. most people mentioning this hav simply said they have seen an obvious, for the better, improvement.

i know in my usage, album art in the webui loads MUCH faster, as a for instance, but i'm not sure i could benchmark it.

MrSinatra
2009-11-13, 07:40
Ok, here go my numbers (this is a Via C3/1GHz, 1GB RAM box):

Before (SBS MySQL):

After (using system's MySQL):

Michael,

i don't do linux, but i'm confused... you have two mysql's? and did you apply the moonbase file to the second one only?

i think your tests should use the same mysql for both, or am i way off here?

mherger
2009-11-13, 08:05
> i don't do linux, but i'm confused... you have two mysql's? and did

Yes. Only the packages use the system's mysqld by default. As I'm running from trunk (svn checkout) I was too lazy configuring this before. I didn't care about the additional memory load.

> you apply the moonbase file to the second one only?

I ran the test against both. The difference for the modified version was below a minute for the grand total, thus I only posted one of them.


Michael

aubuti
2009-11-15, 15:45
some people have talked about doing searches or drilling down to artists or songs via different nav paths.

but if you mean a true benchmark, no. most people mentioning this hav simply said they have seen an obvious, for the better, improvement.

i know in my usage, album art in the webui loads MUCH faster, as a for instance, but i'm not sure i could benchmark it.
After trying it out some more I'd say the mysql tweak may speed up the web ui a little bit on my system, but it's hard to say for sure. Certainly nothing that I would describe as obvious or dramatic.

radish
2009-11-16, 07:28
After trying it out some more I'd say the mysql tweak may speed up the web ui a little bit on my system, but it's hard to say for sure. Certainly nothing that I would describe as obvious or dramatic.

Same here. It's maybe a bit quicker but I didn't consider it slow in the first place.

MrSinatra
2009-11-16, 17:09
it was dramatic for me. how many files do you guys have? i think the effect is linear based on the more you have, the more potent the effect.

radish
2009-11-16, 19:20
it was dramatic for me. how many files do you guys have? i think the effect is linear based on the more you have, the more potent the effect.

Around 22k.

aubuti
2009-11-21, 09:32
it was dramatic for me. how many files do you guys have? i think the effect is linear based on the more you have, the more potent the effect.
That would be straightforward to test. Divide your library up into increasingly large subsets, whether through temporarily re-arranging the folder structure or other means. Then simply compare the responsiveness with different MySQL settings on each subset, being sure to clear the broswer cache in between. To be conclusive it would really help to have some way to quantify what it is you're trying to measure, rather than subjective assessments.

mark f
2009-12-05, 06:10
My results (SheevaPlug, ARM, 1.2 GHz, 512 MB RAM, Ge network) -

Squeezebox Server Status
Version: 7.4.1 - r28947 @ Tue Oct 20 08:02:44 PDT 2009
Hostname: debian
Server IP Address: 127.0.0.1
Server HTTP Port Number: 9000
Operating system: Debian - EN - iso-8859-1
Platform Architecture: armv5tel-linux
Perl Version: 5.10.1 - arm-linux-gnueabi-thread-multi
MySQL Version: 5.1.37-2
Total Players Recognized: 1

Library Statistics
Total Tracks: 4,346
Total Albums: 368
Total Artists: 391
Total Genres: 31
Total Playing Time: 297:06:17

Before:

Music Scan Details
Directory Scan Complete 00:23:20
Playlist Scan Complete 00:02:01
Merge Various Artists Complete 00:01:02
Artwork Scan Complete 00:05:33


The server has finished scanning your music collection.
Total Time: 00:31:56 (Saturday, December 5, 2009 / 5:28 AM)


After:

Music Scan Details
Directory Scan Complete 00:22:57
Playlist Scan Complete 00:02:00
Merge Various Artists Complete 00:01:01
Artwork Scan Complete 00:05:33


The server has finished scanning your music collection.
Total Time: 00:31:31 (Saturday, December 5, 2009 / 6:37 AM)


The scan was not much faster (20 seconds out of 31 minutes is pretty small. HOWEVER, the Web UI is much more responsive. The server page used to take almost a second to show the green screen and 3 seconds to show the left side. Those are almost immediate now (less than a second).

EDIT: In my case, the disk access times may be overwhelming any increased speed for the DB. I moved the collection from a NAS to a USB drive and the scan reduced to ~19 minutes. Soon I'll have a SD card that I can put the collection on and I'll see how that compares.

bobkoure
2009-12-05, 16:04
> I haven't seen many Linux systems so far. Is this more important on Windows I wonder?
I think, to see the difference of a larger innodb_buffer_pool_size in Linux, you also need to set
innodb_flush_method=O_DIRECT
otherwise you're still double-buffering in the OS and MySQL.
See mysqlperformanceblog (http://www.mysqlperformanceblog.com/2007/11/03/choosing-innodb_buffer_pool_size/)

SadGamerGeek
2009-12-05, 18:13
My results

System - WHS (Windows Server 2003), 1GB RAM, Dual Core Pentium (E2180) @2Ghz

Version: 7.4.1 - r28947 @ Tue Oct 20 08:13:15 PDT 2009
Hostname: home-server
Server IP Address: 192.168.0.100
Server HTTP Port Number: 9000
Operating system: Windows Home Server - EN - cp1252
Platform Architecture: 586
Perl Version: 5.10.0 - MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt-log

Total Tracks: 8,627
Total Albums: 719
Total Artists: 322
Total Genres: 78
Total Playing Time: 590:12:39

Before (previous night's scan):

Music Scan Details
Directory Scan (8627 of 8627) Complete 00:08:13
Playlist Scan ( of ) Complete 00:01:11
Merge Various Artists (670 of 670) Complete 00:00:05
Artwork Scan (719 of 719) Complete 00:03:25

The server has finished scanning your music collection.
Total Time: 00:12:54 (Saturday, December 5, 2009 / 04.13)

After (manual scan):

Directory Scan (8627 of 8627) Complete 00:11:23
Playlist Scan ( of ) Complete 00:01:19
Merge Various Artists (670 of 670) Complete 00:00:05
Artwork Scan (719 of 719) Complete 00:03:23

The server has finished scanning your music collection.
Total Time: 00:16:10 (Saturday, December 5, 2009 / 21.23)

So, the initial scan was somewhat slower...

The overnight one is slightly faster than normal though:


Directory Scan (8627 of 8627) Complete 00:07:46
Playlist Scan ( of ) Complete 00:01:08
Merge Various Artists (670 of 670) Complete 00:00:05
Artwork Scan (719 of 719) Complete 00:03:21
The server has finished scanning your music collection.
Total Time: 00:12:20 (Sunday, December 6, 2009 / 04.13)


I was hoping the web UI would be snappier, but I haven't noticed any difference. It took around 4 seconds to load before (in Firefox), and it seems to be exactly the same now.

bstrulo
2009-12-09, 04:44
I changed the mytt. Sorry, I didn't bother to log scan times: they don't usually bother me since I do them at night or when I'm not using the SB. I'll try to note them at some stage but not just now.

But the change in the Web UI was really noticeable. I was so pleased I pushed the innodb_buffer_pool_size up to 128M and still got some noticeable improvements to responsiveness. It changes the Web UI from adequate, to pleasant to use. Particular improvements to search and random mix.

I've voted for the bug.

bstrulo
2009-12-10, 06:44
So I ran some timings and I was quite surprised:

Standard my.tt

Directory Scan (19021 of 19021) Complete 00:51:48
Playlist Scan (10 of 10) Complete 00:00:05
Merge Various Artists (1870 of 1870) Complete 00:01:31
Artwork Scan (1966 of 1966) Complete 00:37:08
Total Time: 01:30:32 (Thu, 10. Dec 2009 / 10:39:08)


Using moonbase's my.tt but with innodb_buffer_pool_size = 128M

Directory Scan (19021 of 19021) Complete 00:52:53
Playlist Scan (10 of 10) Complete 00:00:06
Merge Various Artists (1870 of 1870) Complete 00:01:35
Artwork Scan (1966 of 1966) Complete 00:36:04
Total Time: 01:30:38 (Thu, 10. Dec 2009 / 1:08:14)

So no significant difference (to the accuracy of the benchmark)

I also made some measurements of response times in the web ui and, to the accuracy of my second counting, I couldn't spot any differences. It still "felt" faster but maybe that's placebo.


I also tried moonbase's my.tt unchanged but not for a scan. Again no measurable improvements to the webui.

So, I think it helps, but the numbers don't back me up. Double blind testing anyone :-)

MrSinatra
2009-12-10, 16:14
if i might be so bold, perhaps you didn't delete the cached my.tt each time with a stopped SBS?

somehow i doubt it made no difference. almost everyone has reported faster scanning or faster UI, or normally both. i also doubt the dramatic difference you noted in the UI in your first post was just placebo effect.

dsdreamer
2010-01-01, 11:48
Sorry, I lost my "before" log, but it was a total of 8 minutes and 20 seconds.

The following is my new timing:


Library Statistics
Total Tracks: 7,084
Total Albums: 500
Total Artists: 491
Total Genres: 55
Total Playing Time: 552:49:02

Music Scan Details
Directory Scan (7084 of 7084) Complete 00:02:30
Playlist Scan (2 of 2) Complete 00:00:00
Merge Various Artists (455 of 455) Complete 00:00:08
Artwork Scan (500 of 500) Complete 00:02:08
The server has finished scanning your music collection.
Total Time: 00:04:46 (Friday, January 1, 2010 / 10:17 AM)

The web UI is noticeably faster, and even manipulation of long play lists has become practical for the first time.

Running Windows Vista 64-bit on a Core-2 Duo 2.8GHz CPU with 4 GB RAM installed.

turls
2010-06-02, 14:22
It seems surprising that this thread has not been posted to since January. Are the tweaks in here still valid? I voided my HP WHS warranty to beef it up with a faster proc and more memory and SB is still slow. I guess I won't hurt anything trying these tweaks, but there are only a couple of posts here from WHS users it seems, and none of them showed much improvement.

MrSinatra
2010-06-02, 14:26
the tweaks are for MySQL users. SBS seems to be going to a SQLite by default setup, and so you can't do the tweaks with that. If you do use MySQL for your install, i have not seen the tweaks hurt anyone, while they do seem to help a lot of people.

bobkoure
2010-06-10, 05:19
Are the tweaks in here still valid?
Yes.
These are tweaks to improve the performance of MySQL in the special case of having a database small enough to fit into memory (MySQL is normally used with databases that do not fit). Further, the default settings that SqueezeServer uses for MySQL are for a machine with very little memory.
Starting with SqueezeServer 7.6, if all goes to plan, there will be a choice between SQLite and MySQL as database systems. These tweaks will continue to work - but only if you've chosen / installed MySQL.
Parenthetically, SQLite used to be really slow, which is why SqueezeServer has been using MySQL. But SQLite performance has improved over the last few years. Still not the appropriate choice for a big-ish database, but even with 200K tracks, Squeeze doesn't get into that range. Personally, I use a low-spec (for low power consumption) music server so I'm looking forward to trying SQLite.

Another angle with 7.6 is that, given that the current MySQL settings are for a "minimal" server, and that there will then be SQLite for that minimal server, I'd expect something like these "tweaks" to become the norm (no longer a need to accommodate those minimal boxes as SQLite's there for that).

Hoping that makes it clear...

PS: Just to put the "Squeezeserver's database is small" into perspective, a few years ago I setup a MySQL database that had hundreds of millions of rows.
SqueezeServer's db is more in the tens or hundreds of thousands of rows (depending on the number of tracks)

yerma
2010-08-15, 05:54
I applied the settings on my NAS but I didn't notice any obvious WebUI or scanner speedups. OTOH Controller operation has sped up dramatically, instead of 5-10 sec delays the Controller now displays lists nearly instantly (<1 sec) when searching for music. Quite a significant improvement in usability...

MrSinatra
2010-08-20, 10:18
EVERYONE,

AndyG is willing to make a pref for this, see here:

http://bugs.slimdevices.com/show_bug.cgi?id=14921

however, i don't know how to answer his question... i wish moonbase was still around. we need a DB settings expert to submit to AndyG in that bug exactly what would be best, and exactly what should be changed.

i know i could just copy in moonbases my.tt text, but i can't explain it.

can someone help???

what i'd like to see on the sbs mysql prefs page (short of a self tuning mechanism):

A. default (meaning todays's current shipping sbs settings, ie. the lowest common denominator / sparc nas type numbers)

B. typical 1gig machine

C. typical 2gig machine

D. 3gig

E. 4gig and beyond

---

i don't know if CPU matters to the settings or not, but i know ram size does.

an expert really should answer AndyG's question though. thx!

ps. is it possible to have different "defaults" based on different SBS distrobutions? one default for sparcnas, another for windows sbs?

dsdreamer
2010-08-20, 20:44
I am not the database expert you're looking for, but I suggest applying the KISS principle. There's one set of values (from Moonbase) that has been tested by members of this forum for almost a year now, and we've had overwhelmingly positive reports. I don't suggest making new sets of values, that have yet to be tested and may have unintended consequences.

I know you were looking for a more sophisticated answer than that, but I feel that we're in danger of letting the best be the enemy of the good here. Whatever values are chosen, it is worth using tuning-primer.sh to do a sanity check on them (if you are on a Linux type system) https://launchpad.net/mysql-tuning-primer

andyg
2010-08-21, 10:39
KISS is right. I've just added an option to the Performance settings page called "Database Memory Config". Currently this only works for MySQL and uses Moonbase's config which is listed below. I expect in the future we can also include some SQLite settings when this option is enabled, once we determine what the best settings are.




# $Id$
#
# Squeezebox Server specific MySQL Server config.
# High-memory configuration by Moonbase
# http://forums.slimdevices.com/showthread.php?t=60682

[mysqld]
innodb
skip-locking
long_query_time = 2
log_slow_queries

# If you want to have user permissions - you need to setup a valid user, and
# remove this line below.
skip-grant-tables

basedir = [% basedir %]
datadir = [% datadir %]
tmpdir = [% datadir %]
language = [% language %]
port = [% port %]
socket = [% socket %]
pid-file = [% pidFile %]
log-error = [% errorLog %]
innodb_fast_shutdown = 1
max_connections = 4
thread_concurrency = 4
log-warnings = 0
bind-address = [% bindAddress %]
default-character-set = utf8
default-collation = utf8_general_ci
key_buffer = 16M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K

# InnoDB settings
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 32M
innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

[client]
socket = [% socket %]

andyg
2010-08-21, 10:52
I also added the following pragmas for SQLite when highmem mode is enabled:



PRAGMA cache_size = 20000;
PRAGMA temp_store = MEMORY;


This should let the database use up to 20MB for page cache and store temporary files in memory (used by sorts, joins, etc)

MrSinatra
2010-08-21, 11:10
wow, this is awesome...

what would be killer, would be for someone (not necessarily AndyG) to explain in remarks in the code, what each line is for and some advice on what ranges to use given what hardware one has.

i agree with the KISS thing too, but eventually it would be good to have "differing levels" of settings so one could pick the level that most closely matches their hardware, and i don't see much danger in it if one can simply "switch back" to the default or whatnot.

is this in 7.6 and 7.5.2?

thx AndyG!

Mnyb
2010-08-21, 12:41
This is awesome ! thankyou :)

Btw the differing level would not only match hardware but ones needs .

Me, I want to flat out use my whole little server for sbs ! an easy case.

But it can easily be so that people with powerful machines but not that much focus on sbs want's to run other applications too, then it's great that one can choose wich "size" sbs you want to run :)

Btw is MySQL in7.6 working now ? last time i tried, scan crashed and it reused the old dB since the last successful MySQL scan made with 7.5 beta.
I've not yet figured a way the clear the cache without loosing my SQlite dB and trackstats in the process.

jimzak
2010-08-29, 05:11
Before:

Directory Scan (78706 of 78706) Complete 02:02:54
Playlist Scan (5882 of 5882) Complete 00:08:48
Merge Various Artists (9901 of 9901) Complete 00:05:37
Artwork Scan (10249 of 10249) Complete 01:18:49
Total Time: 03:36:08 (Saturday, August 28, 2010 / 6:06 PM)

After:

Directory Scan (78706 of 78706) Complete 01:46:53
Playlist Scan (5882 of 5882) Complete 00:09:09
Merge Various Artists (9901 of 9901) Complete 00:05:48
Artwork Scan (10249 of 10249) Complete 01:18:38
Total Time: 03:20:28 (Sunday, August 29, 2010 / 7:00 AM)

Not a huge gain but a gain, I suppose.

I'm using a Windows XP nettop with 2 GB RAM and version 7.5.1 of SBS.

MrSinatra
2010-08-29, 10:13
but what about webui? does the interface seem snappier?

ANDYG,

can/will you backport this to 7.5.2?

i think it makes a lot of sense to get this mod into what looks to be the last "mysql only" SBS. thx!

andyg
2010-08-29, 16:33
OK it's pretty self-contained so I can backport it to 7.5.

MrSinatra
2010-08-29, 17:49
thx andy! i'll look for it this week.

Archimago
2010-08-29, 19:07
Thanks Moonbase for this thread.

For folks with a relatively powerful Squeezebox Server system and **RAM to burn** :-), I've attached my settings for "my.tt". For those running into trouble, rebooting and doing a "Clear library and rescan everything" will avoid database issues.

On my system (4G RAM, Opteron 165 Dual Core @2GHz, SB Server 7.5.1, 53K songs on 1.5TB WD Caviar Green SATA, 600x600+ artwork, 70% FLAC, ~200 24/96+ albums/LP rips), a full rescan was >30% faster and day-to-day use appears speedier. Web interface quite snappy and CLI apps like iPeng & SqueezePad at times instantaneous. Details below...

As usual, YMMV :-)
Archimago

-----------------------------
Tests (these are all "Clear library and rescan everything" times after a reboot):

Squeezebox Server Information:
Version: 7.5.1 - r30836 @ Tue Jun 1 06:02:45 PDT 2010
Operating system: Windows 2008 Server R2 - EN -
Platform Architecture: 586
Perl Version: 5.10.0 - MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt-log

Default my.tt: (sorry, didn't copy and paste detailed timings)
Total Time: 1:55:05

Moonbase's my.tt:
Total Time: 1:40:20

Moonbase's my.tt + 300M innodb_buffer_pool_size:
Directory Scan (53293 of 53293) Complete 00:53:49
Playlist Scan (3 of 3) Complete 00:00:00
Merge Various Artists (3785 of 3785) Complete 00:01:27
Artwork Scan (3928 of 3928) Complete 00:40:46
Total Time:01:36:02

ArchMega500 my.tt (500M innodb_buffer_pool_size + key_buffer, etc. tweaks):
Directory Scan (53293 of 53293) Complete 00:31:25
Playlist Scan (3 of 3) Complete 00:00:00
Merge Various Artists (3786 of 3786) Complete 00:01:29
Artwork Scan (3929 of 3929) Complete 00:40:30
Total Time:01:13:24

Observation:
1. With just my FTP Server and Apache running in the background unused, Windows reports physical memory usage not to exceed 2.2GB through the scan. Memory usage drops thereafter in day-to-day use.
2. I don't see much multithreading going on at all, Directory Scan uses about 30-40% CPU, Artwork Scan only uses up to 50% of the dual cores. Therefore, HD speed, MHz/GHz speed more important than # cores for the scanning process. Artwork Scan probably CPU-bound, I assume this is where all the artwork resizing being done.
3. IMO key_buffer and table_cache just as significant as innodb_buffer_pool_size. I actually did not notice any difference between 300M to 500M for the pool_size but decided to leave it as 500M to give the database room to grow.
4. An interesting related article:
http://www.debianhelp.co.uk/mysqlperformance.htm

agonynine
2010-09-09, 03:48
KISS is right. I've just added an option to the Performance settings page called "Database Memory Config". Currently this only works for MySQL and uses Moonbase's config which is listed below. I expect in the future we can also include some SQLite settings when this option is enabled, once we determine what the best settings are.




# $Id$
#
# Squeezebox Server specific MySQL Server config.
# High-memory configuration by Moonbase
# http://forums.slimdevices.com/showthread.php?t=60682

[mysqld]
innodb
skip-locking
long_query_time = 2
log_slow_queries

# If you want to have user permissions - you need to setup a valid user, and
# remove this line below.
skip-grant-tables

basedir = [% basedir %]
datadir = [% datadir %]
tmpdir = [% datadir %]
language = [% language %]
port = [% port %]
socket = [% socket %]
pid-file = [% pidFile %]
log-error = [% errorLog %]
innodb_fast_shutdown = 1
max_connections = 4
thread_concurrency = 4
log-warnings = 0
bind-address = [% bindAddress %]
default-character-set = utf8
default-collation = utf8_general_ci
key_buffer = 16M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K

# InnoDB settings
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 32M
innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

[client]
socket = [% socket %]


This is slightly different from Moonbase's final my.tt recommendations, where he used key_buffer_size instead of key_buffer etc....



# Some tweaks by Moonbase, recommended for systems with 1GB+ RAM

# key_buffer: cache index blocks for MyISAM, temp disc tables for InnoDB
key_buffer_size = 16M
# sort_buffer_size: improve ORDER BY and GROUP BY
sort_buffer_size = 2M
# join_buffer_size: improve full JOINs (non-indexed)
join_buffer_size = 2M
# read_buffer_size: don't adjust too high, just for full scans
read_buffer_size = 512K
# read_rnd_buffer_size: really improve ORDER BY
read_rnd_buffer_size = 2M


Is there any reason for this?

andyg
2010-09-09, 05:31
I've updated my-highmem.tt with these newer values.

MrSinatra
2010-09-15, 12:26
OK it's pretty self-contained so I can backport it to 7.5.

Andy,

just installed 7.5.2 to my Dad's system, but i don't see it... is it there yet? if yes, where? can it be in tonights nightly if not?

thx.

MrSinatra
2010-09-20, 21:42
sorry to bug you Andy, but whats the word?

andyg
2010-09-20, 21:55
Sorry haven't got around to it yet, will do it tomorrow.

MrSinatra
2010-09-20, 21:58
Sorry haven't got around to it yet, will do it tomorrow.

thx, u rock! so tomorrow's nightly then... looking forward to it.

andyg
2010-09-21, 07:48
Done, it's in 7.5 in r31367.

rolski
2010-10-25, 03:19
Any experiences with using these newly added options ?

Recommended settings ?

MrSinatra
2010-10-25, 03:22
what do you mean?

i just enable it via the webui settings>advanced page.

i think it works great as is, but it prob could use more than just on/off, (to be even better).

its in 7.5.2b and 7.6b

rolski
2010-10-28, 13:31
Apologies for the delayed reply....
I have a 1GB RAM ReadyNAS NV+ with the following library stats :-

Total Tracks: 22,261
Total Albums: 2,116
Total Artists: 3,373
Total Genres: 29
Total Playing Time: 1751:33:33

Music Scan Details
Directory Scan (22268 of 22268) Complete 02:38:01
Playlist Scan Complete 00:00:00
iTunes Import (21651 of 21651) Complete 00:47:45
iTunes Playlist Import (6 of 6) Complete 00:02:23
Merge Various Artists (504 of 504) Complete 00:01:19
Artwork Scan (2116 of 2116) Complete 00:08:00
The server has finished scanning your music collection.
Total Time: 03:37:28 (Thursday, 28 October 2010 / 06.44)Now, with the new option - which I understand is directed towards reserving more RAM for scans etc, the database memory config can be set to "high", but I also assume further tweaking of Server Priority and Scanner Priority, in conjunction with the memory setting, may reap additional rewards.
My previous question, perhaps not well worded, should have been more along the lines of asking whether anyone's found a good / ideal / bad / non-workable (etc) balance of all 3 settings....?

DLloyd
2010-12-10, 04:33
Hello,

I have been reading this thread and this evening I did the following on my Debian server. This completely killed the Squeezebox Server:

Stopped SqueezeboxServer + MySQL

Deleted both:

/etc/mysql/my.cnf
/var/lib/squeezeboxserver/cache/my.cnf

I edited:

/usr/share/squeezeboxserver/MySQL/my.tt with the new moonbase settings (my.tt.txt) adding the following to the file:

max_allowed_packet = 1M
table_cache = 64
net_buffer_length = 8K
key_buffer_size = 16M
sort_buffer_size = 2M
join_buffer_size = 2M
read_buffer_size = 512K
read_rnd_buffer_size = 2M
innodb_buffer_pool_size = 40M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

Rebooted. Nothing. I thought that my.cnf was somehow regenerated from my.tt??

So, I created a new /etc/mysql/my.cnf and used the values below (which are also from this thread). These are for a Windows install, so I changed basedir="/" and datadir="var/lib/mysql"

This doesn't work either and I have no way of knowing my original my.cnf settings....

************************************************** ****************************

[client]

port=3307

[mysql]

default-character-set=latin1

[mysqld]

port=3307

basedir="R:/Program Files/MySQL/MySQL Server 5.1/"

datadir="R:/Program Files/MySQL/MySQL Server 5.1/Data/"

default-character-set=latin1

default-storage-engine=INNODB

sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_E NGINE_SUBSTITUTION"

max_connections=100

query_cache_size=128M

table_cache=256

tmp_table_size=17M

thread_cache_size=8

myisam_max_sort_file_size=100G

myisam_sort_buffer_size=34M

key_buffer_size=25M

read_buffer_size=64K
read_rnd_buffer_size=256K

sort_buffer_size=256K

#skip-innodb

innodb_additional_mem_pool_size=2M


innodb_flush_log_at_trx_commit=1


innodb_log_buffer_size=1M


innodb_buffer_pool_size=96M


innodb_log_file_size=8M


innodb_thread_concurrency=8

innodb_data_file_path=ibdata1:10M:autoextend

MrSinatra
2010-12-10, 11:22
lloyd,

the option is now a built in setting in 7.5.2beta settings. very stable, i recommend using that.

DLloyd
2010-12-10, 20:58
Right now my system is completely trashed.

I have had to remove/purge squeezeboxserver.

bobkoure
2010-12-14, 12:15
After doing some reading on MySQL performance tuning, it looks like innodb_buffer_pool, if memory permits, should be set to 110% of the database size.
If you have a MySQL user setup you can use a SQL query to get the size of the DB. Otherwise, just look at the size of the ibdata file. In windows, that's in ...\Cache\MySQL\.
So, for instance, my ibdata file shows as 247,808K (242M), so I set innodb_buffer_pool to 267M.

efalarde
2010-12-19, 08:49
Scans had grown to 9 hours and it was now taking more than 50 seconds for new items added to a playlist to appear in my own server system!

Thanks to new parameters, scanning is back to 2 hours (42000 tracks) and basic operations are now typically back to < 1s.

Using HP MediaSmart (old gen) that I have upgraded to 2Gb.

marsboer
2010-12-22, 03:45
Is it just me or has the option to enable 1GB+ memory tuning for MySQL disappeared from the final 7.5.2 release (using the debian version from repository)?
The tuning is also not present in my.tt as it was in the betas.

JJZolx
2010-12-22, 04:22
Is it just me or has the option to enable 1GB+ memory tuning for MySQL disappeared from the final 7.5.2 release (using the debian version from repository)?
The tuning is also not present in my.tt as it was in the betas.

No, it's not you. It's a pretty screwed up situation. They released 7.5.2 only with new firmware. If you were running 7.5.2 beta for any of the updates or bug fixes then you _don't_ want to install the 7.5.2 release version. Go back to the beta builds, which are now versioned 7.5.3.

MrSinatra
2010-12-22, 13:02
Is it just me or has the option to enable 1GB+ memory tuning for MySQL disappeared from the final 7.5.2 release (using the debian version from repository)?
The tuning is also not present in my.tt as it was in the betas.

ANDYG, is this true? has anyone checked the 7.5.3 beta to see if its in that?

mherger
2010-12-22, 13:38
> ANDYG, is this true? has anyone checked the 7.5.3 beta to see if its
> in that?

Yes, that's true. If you read the announcement then you'll see that the
released 7.5.2 was basically a firmware release only. Don't panic: if
you've been using 7.5.2 before, you should upgrade to the 7.5.3 nightlies,
which include all of the previous 7.5.2 changes.

--

Michael