Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 6 123 ... LastLast
Results 1 to 10 of 53
  1. #1
    Junior Member
    Join Date
    Aug 2010
    Posts
    24

    Erland's Plug-ins, Large Collections, and Hardware Requirements

    I have been an extremely satisfied VortexBox Appliance (VBA) owner since December 2009. In that time I've built an extensive music library while modifying both the hardware and software of my machine incrementally over the years. I've succeeded in converting several friends into avid VB enthusiasts.

    As my collection grew, I found that my setup was struggling to handle database queries. Playback was fine, as long as I used mechanisms to locate discs that didn't result in substantial database operations. Mostly I would do an artist or an album query. A song query would invariably hang or crash LMS. I have an extensive classical collection and I spent a great deal of effort entering and maintaining the requisite metadata. I was looking forward to having the ability of, say, pulling up all of my recordings of Bruckner's Seventh Symphony or listing all of the recordings of a given pianist by composer. Erland Isaksson's LMS plugins (Custom Browse, Custom Scan, Mixed Tags, etc) seemed ideally-suited for this task. Unfortunately, whenever I tried to run these queries they would either time out or LMS would crash.

    I started a correspondence with Erland who couldn't have been more helpful. I sent him log files, database files, and several other diagnostics. I began noticing that LMS was also spontaneously dying overnight with out-of-memory errors. My machine was a standard VBA with 1 GB of RAM and an original 2TB of disc space expanded to 5TB, with my collection totaling approximately 120,000 tracks. I also keep an MP3 mirror and run Subsonic. There have been many threads over the years in the VortexBox forums inquiring about whether 1GB of memory is sufficient. The standard answer was that VB used far less than 1GB and that more memory shouldn't lead to increased performance.

    After getting absolutely nowhere after many months of happy listening but growing frustration with my limited capabilities, I decided to build a new server box. I bought a new Lenovo box from Fry's for $299. It had 6GB of memory (massive overkill), a 2.8 Ghz dual-core AMD processor, and a 500GB drive preloaded with Windows 7. It took me about $300 and less than 10 minutes to install two new 3TB drives (pulling and putting aside the Windows disc) along with a USB3 card. In about an hour I had a fully-functioning VB. In about a 24 hours, all of my music and settings were transferred from the old box.

    My results have been jaw-dropping. Not only are all of my Custom Browse queries running, they usually return results within one or two seconds. There is clearly significant caching going on, so after the first query things go much more swiftly thereafter. I can search by song now, where on my old box a song query would have been equivalent to hitting the “Stop LMS” button. And I can do all of this with music playing, literally without missing a beat. I've been looking at the System Status figures while I run queries. In steady state, “kernel and applications” use about 1.3 GB of memory. The rest is spent on caching, buffers, and disk swap.

    My suspicion is that while LMS and Subsonic can run on a VBA with less than 1GB of RAM, a large collection combined with complex queries pushed the hardware over the brink. I didn't structure my upgrade as a controlled experiment. Perhaps merely increasing RAM to 2GB on my VBA would have worked nearly as well (none of the diagnostics ever suggested that my VBA was CPU-bound). I just wanted to pass on that for users of Erland's LMS plugins with large libraries, the current VBA hardware configuration may not be insufficient to utilize the full capabilities of the LMS package.

    The good news is that I am absolutely thrilled with my new VB setup. I wish realized sooner that my problems were hardware-related. I'm posting this in the hope of helping others who may be struggling with similar issues.

  2. #2
    Senior Member vagskal's Avatar
    Join Date
    Oct 2008
    Location
    Sweden
    Posts
    643
    Thanks for sharing.

    Just to be sure I understand: Vortex Box runs under Unix and you are using Unix in the new setup, not Win 7? (Sorry if this is a basic question, but I have not used Vortex Box. I thought it was a Linux build of some kind incorporating LMS.)
    2 x SB3 (wired), Receiver (wired), Boom (wireless), Controller, iPeng on iPhone 4 & iPad, muso on remote computer running Win 7 64-bit | 7.7.3 on Win XP

  3. #3
    Junior Member
    Join Date
    Aug 2010
    Posts
    24
    Quote Originally Posted by vagskal View Post
    Thanks for sharing.

    Just to be sure I understand: Vortex Box runs under Unix and you are using Unix in the new setup, not Win 7? (Sorry if this is a basic question, but I have not used Vortex Box. I thought it was a Linux build of some kind incorporating LMS.)
    Vortexbox is a package based on Fedora Linux which includes LMS and a number of related utilities.

    http://vortexbox.org/content/134-About-VortexBox

    I have not run LMS under Win 7, but the issues I discussed in my post may pertain as well. The point is if you are working with a relatively low-memory underpowered machine, a hardware upgrade (particularly, a memory upgrade) may yield substantial results in Custom Browse performance.

  4. #4
    Senior Member vagskal's Avatar
    Join Date
    Oct 2008
    Location
    Sweden
    Posts
    643
    The point is if you are working with a relatively low-memory underpowered machine, a hardware upgrade (particularly, a memory upgrade) may yield substantial results in Custom Browse performance.
    Thanks. Yes, I got your point. Since I already have a Win XP PC with rater decent specs, a fairly large collection, Custom Browse, Custom Scan _and_ the performance issues you described I just wondered if switching to Linux could give a performance boost.
    2 x SB3 (wired), Receiver (wired), Boom (wireless), Controller, iPeng on iPhone 4 & iPad, muso on remote computer running Win 7 64-bit | 7.7.3 on Win XP

  5. #5
    Senior Member lrossouw's Avatar
    Join Date
    Jan 2006
    Location
    Cape Town, South Africa
    Posts
    1,034
    I had a PC loaded XP as my server. The hardware was (and is) dying a slow death. Some mmemory modules died and I was left with 1GB. Something else went wrong (can't remember what) and I decided to switch to Ubuntu on this machine. This is my first Linux experience. I do believe that the Squeeze experience is much better. More stable and more responsive on my Touch. This is using the same hardware and music. Also using erland's plugins on both setups.
    Louis
    Last.fm

  6. #6
    Junior Member
    Join Date
    Aug 2010
    Posts
    24
    Quote Originally Posted by lrossouw View Post
    I had a PC loaded XP as my server. The hardware was (and is) dying a slow death. Some mmemory modules died and I was left with 1GB. Something else went wrong (can't remember what) and I decided to switch to Ubuntu on this machine. This is my first Linux experience. I do believe that the Squeeze experience is much better. More stable and more responsive on my Touch. This is using the same hardware and music. Also using erland's plugins on both setups.
    My experience has been that Linux will give you much more bang for your buck on the same hardware vs. Windows, though to be fair I haven't done comparisons vs. Windows 7. The Vortexbox distribution is designed to be run on very modest hardware. The creators purposely paired down the Fedora distribution for this purpose, excluding X Windows, the Gnome Desktop environment, and scores of other things, all of which can be installed by the user if they desire. The distribution is designed to be run "headless", without monitor or keyboard. There is a built-in web interface that can be accessed via any other machine on the same network that will suffice most users for normal use and maintenance. Users can also ssh in (via Putty on Windows or Terminal on the Mac) if a higher degree of control is required. I've only required a monitor/keyboard for initial installation.

    If you want to do an experiment and your machine is relatively easy to work with, remove your internal disc and place any internal disc inside in it's place, even a relatively small/cheap one. Download the Vortexbox ISO, burn to CD, and install it in the box. Put your old Windows disk in another bay, an external enclosure, or just attach it via a USB/SATA connector. Linux readily deals with Windows NTFS discs. Mount your old music directory somewhere in the /storage/music directory (or wherever you've configured LMS to look), set up Erland's plugins properly and do a rescan. Play around and test out how things work for you. If you don't like it any better, just put your windows disc back in the box. If you do like it, then you have your answer.

  7. #7
    Senior Member
    Join Date
    Apr 2007
    Posts
    289
    This thread gives me something to look into. I only have ~21,000 tracks, but I use many of Erland's plugins (Multi Library, Custom Scan, TrackStat), and while playing songs is fine, the database queries are dreadfully slow. I'm actually in the process right now of trying to figure out which plugin (if any) is responsible. My LMS server is running on a Mac Mini 1.83 Ghz Core 2 Duo with 3 GB of RAM. Right now any scan or server start up takes at least an hour.

  8. #8
    Senior Member erland's Avatar
    Join Date
    Dec 2005
    Location
    Sweden
    Posts
    11,290
    Quote Originally Posted by prabbit View Post
    This thread gives me something to look into. I only have ~21,000 tracks, but I use many of Erland's plugins (Multi Library, Custom Scan, TrackStat), and while playing songs is fine, the database queries are dreadfully slow. I'm actually in the process right now of trying to figure out which plugin (if any) is responsible. My LMS server is running on a Mac Mini 1.83 Ghz Core 2 Duo with 3 GB of RAM. Right now any scan or server start up takes at least an hour.
    Sometime when you have the time, it would be great if you enabled debug level logging on "plugin.trackstat" and "plugin.customscan" in LMS Settings/Advanced/Logging and also select the "Save logging settings for use at next application restart" and then restart LMS and send me the server.log after the server has started completely, the extra debug logging will indicate how long time each phase of the startup takes and this will help me see if there is a certain part of the startup that takes time or if it's evenly distributed. After the server has started, remember to reset the log settings, else you are going to get gigantic log files.
    Erland Isaksson (My homepage)
    Developer of many plugins/applets
    Starting with LMS 8.0 I no longer support my plugins/applets (see here for more information )

  9. #9
    Senior Member
    Join Date
    Apr 2007
    Posts
    289
    The culprit is Trackstat. More specifically I think it is TrackStat and Musicbrainz IDs. I removed the plugins in this order: Custom Scan, Multi Library, TrackStat. After removing Custom Scan and Multi Library it still took about one hour to start up the server. After removing TrackStat it was about one minute to start the server. It was also one minute after I reinstalled Custom Scan and Multi Library without installing TrackStat.

    Before turning on logging I want to look into some older posts related to TrackStat and MusicBrainz. I seem to recall that there is something. My nightly TrackStat backup file has gotten quite large (~40 MB). I haven't looked at it yet, but I'll look into it as well.

  10. #10
    Senior Member pallfreeman's Avatar
    Join Date
    Apr 2010
    Location
    Squeezebox Graveyard
    Posts
    500
    Quote Originally Posted by prabbit View Post
    My LMS server is running on a Mac Mini 1.83 Ghz Core 2 Duo with 3 GB of RAM. Right now any scan or server start up takes at least an hour.
    Just a thought: do you have dbhighmem enabled? (Settings->Advanced->Performance)
    Don't push your love too far
    Your wounds won't leave a scar
    Right now is where you are
    In a broken dream.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •