Announcement

Collapse
No announcement yet.

Server performance questions, ideas, considerations....

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Server performance questions, ideas, considerations....

    We all know SqueezeCenter will run OK on very modest hardware. But what can we do to design a BETTER server for our uses?

    First off, we have to ask: What makes for a "good" SqueezeCenter server?

    It shouldn't crash, certainly.
    It should serve streams to the devices we connect to it smoothly, without stalling, pausing, stuttering, etc.
    It should execute searches quickly.
    It should run a scan quickly.

    With small msuic collections, a proper Squeezecenter installation seems to run fine on just about any fairly modern system.

    But I have over 5,000 cds ripped to WMA lossless, and performance starts to be an issue with collections of this size. To wit:

    **Searches aren't fast as I'd like**
    **A complete scan of the library takes much longer than I'd like**
    **Occasionally, when the server is busy doing a search, etc, it causes interruptions in streams when serving multiple players.**

    So how to build a "better" server?

    There's a lot to consider. CPU speed, memory size & speed, platform I/O performance, disk capacity, disk performance, LAN performance, OS overhead, application tuning......

    There's some things I need to know about SqueezeCenter - in particular, how much can it take advantage of multiple CPU cores? In other words, how much of the Squeezecenter and MYSQL code is optimized for multi-threading? It would be good to hear from Logitech about this!

    I know the Windows PERL interpreter is a multi-threaded design. From my own observations, however, it doesn't look like other parts of the software take very good advantage of multi-core CPUs.

    I have SqueezeCenter 7.2.1 running on a quad core Intel CPU. When I do a music search or rescan the folders, I see a lot of activity on just one CPU core- I don't see much of the workload being spread out over avaialble resources. This is probably why it takes hours to do a directory rescan.

    Also, it seems that both stream serving and music searches run on the same single CPU core. So, when I am in the living room running a search for a particualr artist, the person listening in the den gets "stutters" in the stream. (My LAN is a 1 GB ethernet LAN, not Wifi, and very lightly loaded so I know it's NOT a network capacity issue. And the NIC in my Squeezecenter server is a good 1000 MBPS PCIe device.)

    I wonder if there is a way of tuning Squeezeserver to make it run better- is it possible to manually assign differing CPU affinities to various component processes in order to improve performance?

    I also wonder if the instance of MYSQL could be tuned for higher performance. Is there a way to tell how large the MYSQL instance is- and to size RAM accordingly? I'm currently using 32 bit XP with 2 GB of RAM, but if it would run faster, I could go to 8 GB of RAM and a 64 bit OS like XP64 or Ubuntu. Some guidance from any MYSQL DBA types out there or from Slim Devices HQ on this would be appreciated.

    Does MYSQL take advantage of all available RAM - is it possible to tune MYSQL to minimize disk use to eliminate the bottleneck represented by HDD I/O?

    Also, are there any perfomance metrics to help decide if SqueezeCenter runs faster on Ubuntu or XP?

    #2
    Server performance questions, ideas,considerations....

    > There's some things I need to know about SqueezeCenter - in particular,
    > how much can it take advantage of multiple CPU cores? In other words,


    In your case: a lot. WMA lossless can't be played by our hardware. Thus it has to be transcoded on the server. Which means if you have multiple cores, one can handle the transcoding, while the other is available for SC etc.

    > how much of the Squeezecenter and MYSQL code is optimized for
    > multi-threading?


    SC isn't multi-threaded. That said MySQL and SC are different processes, thus can run on different cores. And the scanner is its own process as well.

    > I have SqueezeCenter 7.2.1 running on a quad core Intel CPU. When I do


    Now that should be powerful enough for anything we're doing...

    > Also, it seems that both stream serving and music searches run on the
    > same single CPU core.


    Correct: as mentioned SC is single threaded.

    > I wonder if there is a way of tuning Squeezeserver to make it run
    > better


    Convert your collection to flac, which can be played without transcoding. If you play four streams, and all have to be transcoded, this will put heavy load on your machine.

    > of RAM, but if it would run faster, I could go to 8 GB of RAM and a 64
    > bit OS like XP64 or Ubuntu.


    I think it's generally agreed upon Linux being a better platform for SC than Windows.

    --

    Michael
    Michael

    "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
    (LMS: Settings/Information)

    Comment


      #3
      Linux vs. XP

      >>I think it's generally agreed upon Linux being a better platform for SC than Windows.<<

      Why?

      My experience tends to be that there is a general "anything MUST be better than Microsoft" feeling among many, but I usually don't like to draw conclusions until I have some data.

      Comment


        #4
        Flac

        >>Convert your collection to flac, which can be played without transcoding. If you play four streams, and all have to be transcoded, this will put heavy load on your machine.<<

        I had thought to do this anyway, because "fast forward" isn't available for the transcoded formats. But it hadn't occurred to me that transcoding must be quite a burden on the system. An excellent idea. I have dBPowerAmp's converter, that should do the trick.

        Thanks

        Comment


          #5
          Server performance questions, ideas,considerations....

          >>> I think it's generally agreed upon Linux being a better platform for
          > SC than Windows.<<
          >
          > Why?


          I don't know. Maybe because perl has grown on Linux/Unix systems much longer than on Windows.

          > My experience tends to be that there is a general "anything MUST be
          > better than Microsoft" feeling among many, but I usually don't like to
          > draw conclusions until I have some data.


          It's not just anti-MSism, believe me. If you don't have a problem running a Linux box, give it try. You can decide yourself.

          --

          Michael
          Michael

          "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
          (LMS: Settings/Information)

          Comment


            #6
            Originally posted by mherger View Post
            >>> I think it's generally agreed upon Linux being a better platform for
            > SC than Windows.<<
            >
            > Why?


            I don't know.
            Michael, you seem to be a smart guy, but this exchange says it all as far as I'm concerned.

            What if your barber recommended you use a straight razor to shave your neck because it's better? Or your mechanic suggested you can easily install a new engine in your car by yourself?

            Stupid examples perhaps. It's one thing to work with something every day, but another matter entirely if one doesn't. There seems to be pack of geeks on these boards who think the general public has any interest or capability to use Linux. As a self-appointed spokesman, I would like to say here and now:

            1. SC runs great on Windows. So do most all other programs, for that matter. How could SC run "better" on something else - either something runs adequately or it doesn't.

            2. Who cares if it also runs on something else? I know nothing about that "something else" and don't care to know anything.

            3. Who cares if linux is free? In the greater scheme of things, a copy of Windows is cheap. And if one is pressing an old PC into service for SC, it likely already has an operating system on it, so indeed, it is free because it is already there.

            4. My spare time is limited, and I don't want to spend it figiting with something I could have otherwise avoided by using Windows.

            By the way, there seem to be far more posts from linux users having problems, than similar things from Windows or Apple users. If you consider what a small percentage of users actually using linux, I would observe that it may be a poor choice for running SC, people who do so seem to have nothing but problems.

            I'll apologize for hijacking this thread in the DIY area, where obviously people are going to be more hands-on geeky than elsewhere. But i wanted to get the point across, it's tiresome to read the constant pushing of linux to a disinterested audience.

            Comment


              #7
              Server performance questions, ideas,considerations....

              >> >>> I think it's generally agreed upon Linux being a better platform
              >> for
              >> > SC than Windows.<<
              >> >
              >> > Why?

              >>
              >> I don't know.

              >
              > Michael, you seem to be a smart guy, but this exchange says it all as
              > far as I'm concerned.


              Ok, I should have said: nobody has proved the contrary? Everybody who has done has come to this conclusion? Or the Perl implementation on Windows sucks?

              I don't know the reason for this. I never blamed Windows for it, and I was simply honest, telling you I didn't know the reason for this difference. But it's the way it is. IMHO it's a fact. Give it a try.

              > 1. SC runs great on Windows. So do most all other programs, for that
              > matter. How could SC run "better" on something else - either something
              > runs adequately or it doesn't.


              I'm only talking about performance.

              > 3. Who cares if linux is free?


              Nobody. Don't know what this has to do here.

              milosz was asking about whether he should give Linux a try. I'd say yes, because he obviously can and was considering it. I'd never suggest my father (or you) to use Linux for the sake of it.

              --

              Michael
              Michael

              "It doesn't work - what shall I do?" - "Please check your server.log and/or scanner.log file!"
              (LMS: Settings/Information)

              Comment


                #8
                Originally posted by Goodsounds View Post
                Michael, you seem to be a smart guy, but this exchange says it all as far as I'm concerned.

                What if your barber recommended you use a straight razor to shave your neck because it's better? Or your mechanic suggested you can easily install a new engine in your car by yourself?

                Stupid examples perhaps. It's one thing to work with something every day, but another matter entirely if one doesn't. There seems to be pack of geeks on these boards who think the general public has any interest or capability to use Linux. As a self-appointed spokesman, I would like to say here and now:

                1. SC runs great on Windows. So do most all other programs, for that matter. How could SC run "better" on something else - either something runs adequately or it doesn't.

                2. Who cares if it also runs on something else? I know nothing about that "something else" and don't care to know anything.

                3. Who cares if linux is free? In the greater scheme of things, a copy of Windows is cheap. And if one is pressing an old PC into service for SC, it likely already has an operating system on it, so indeed, it is free because it is already there.

                4. My spare time is limited, and I don't want to spend it figiting with something I could have otherwise avoided by using Windows.

                By the way, there seem to be far more posts from linux users having problems, than similar things from Windows or Apple users. If you consider what a small percentage of users actually using linux, I would observe that it may be a poor choice for running SC, people who do so seem to have nothing but problems.

                I'll apologize for hijacking this thread in the DIY area, where obviously people are going to be more hands-on geeky than elsewhere. But i wanted to get the point across, it's tiresome to read the constant pushing of linux to a disinterested audience.
                Personal experience/observations.

                My SC runs all the time. My current SC is running on a low wattage fan-less "via C7 1.3Ghz" (i386) based platform with 1G RAM. The OS is Unix based FreeNAS with SlimNAS installed (thanks again Michael). I currently have 32k song titles and 5 SB's throughout the house. Performance is excellent. The host is up 7/24 and only gets rebooted during power outages or when I am upgrading SC, which once every several months or so. The platform is also used as the household NAS.

                My old SC platform was a AMD 64 based system running XP. Performance was similar, but the node required rebooting every couple of weeks.

                Linux/Unix seems to make better use of system resources and has less overhead than XP. Linux has become pretty much the standard OS for WEB based host and application servers. This is mainly because of lower initial purchase and reoccurring operational costs. Also less system overhead adds up when multiplied by a hundred or a thousand systems.

                I work in the telecom industry where SUN/Solaris is the standard application server platfornm. We have been seeing a lot of interest in linux over the past few years and very little interest none in MS 2003 Server.

                Having said that, if you have a relatively new XP pc in your home I am sure it will have plenty of juice to run SC in the background for your application. Just be prepared to reboot it once or twice a month, especially if it is being used as the household pc as well (memory leaks?). If it gets turned on only when you want to listen to music, or someone is surfing the net, I see not problems using XP based pc as a SC host at all.

                my 2 cents.
                Home Office
                SB2->Benchmark DAC-1-> Bryston P-25, preamp -> Carver M1.0t Amp->PMC TB2
                Home Theater System#1
                SB2->Anthem AVM60->Bryston 9B ST -> PSB Stratus Goldi
                /Home Theater System #2/ LazyEye Bar
                Pi3 w/7" screen/HiFiBerry DAC>Outlaw 976-> Bryston 3B ->Klipsch La Scala's, 2x Bryston 4B (mono) EV 18" subwoofers
                Bedroom System
                SB2-> Sony BoomBox
                Rear Deck/Patio
                Pi3 HiFiBerry DAC --> Crown XLS 1500-> PSB Mini's,
                Kitchen
                Pi3 HifiBerry DAC --> Crown XLS 2502-> Polk Ceiling Speakers

                Comment


                  #9
                  Michael:

                  "Ok, I should have said: nobody has proved the contrary?"

                  Have you ever seen the Monty Python argument sketch? You might like it.

                  "Or the Perl implementation on Windows sucks?"
                  "I'm only talking about performance."

                  If these things are true, all the more kudos are due to you and your colleagues for overcoming these heavy disadvantages. Your program runs responsively and for weeks on end in Windows on my old and mediocre PC. Thanks to everyone for making it happen

                  "Who cares if linux is free?
                  Nobody. Don't know what this has to do here"

                  I hear this cost matter frequently from Linux advocates - you never have? Yet these same people will spend hours/days trying to fix something simple that should work but doesn't. Is that something you've not heard of either?

                  My free time is too precious, I'll happily pay the 100 bucks and add countless hours to my free time.

                  Again, I love your products and I enjoy trying to inject some reality into discussions that look like they may need it. A sincere thanks to you for what you do, and hey, start pitching Windows every now and then. Bill and Steve will love you for it.

                  Comment


                    #10
                    Hi sfraser-

                    I found your comments interesting and I'm glad your setup is working well for you. The motivation for my comments was really best stated in my last sentence. There are some number of mantras that get chanted in these forums that just get a bit old and that's why I spoke up.

                    I run Slim Server (no desire to upgrade, it runs fine as is) and music on a 7 year old PC with 256M of memory and Win XP. It's also used for other things. My music collection is about one-half the size of yours. Just as you described your old machine, mine gets bogged down every few weeks and I reboot it. I don't find that to be a problem.

                    I understand that the world of IT outside the home is a completely different one. But so too are the resources. Linux is a big deal in industrial/commercial/internet server settings, but it is not a big deal in the home. And I doubt it ever will be.

                    Squeeze products run in homes, not in server farms, and I think the company needs to exert a much greater effort to make its products, and these forums, more friendly to Joe Public. Joe Public has an old XP machine, doesn't know or care what linux is, and he just wants to listen to his music. Technical stuff is a turn-off for Joe.

                    Good luck to you, thanks for your comments.

                    Comment


                      #11
                      To help illustrate "the other side of the fence"...

                      Originally posted by Goodsounds View Post
                      SC runs great on Windows. So do most all other programs, for that matter. How could SC run "better" on something else - either something runs adequately or it doesn't.
                      Under most circumstances, Windows will have software installed on it that may lead to trouble--anti-virus software for example. So with Windows, there's a higher risk of out-of-the-box failure, and having to dink around with your machine to get it working. With Linux, you have a much greater chance of having it just work without any hassles.

                      Originally posted by Goodsounds View Post
                      There are some number of mantras that get chanted in these forums that just get a bit old and that's why I spoke up.
                      One man's mantra is another man's good advice. Good advice is worth repeating.

                      Originally posted by Goodsounds View Post
                      I run Slim Server (no desire to upgrade, it runs fine as is) and music on a 7 year old PC with 256M of memory and Win XP
                      No doubt many others do too. However, I'd recommend replacing the hard drive on such an old machine, which is where it gets tricky for many non-techie users. Most users have never installed an operating system, and replacing a hard drive necessitates that. So if you're replacing the hard drive, then there is no difference to non-techies between Windows and Linux because they both are just as difficult to install. Windows gets its easy-to-use reputation because nobody ever installs it, and Linux gets its hard-to-use reputation because installing it is the very first thing you have to do with it. But other than that they're really not that different for running a simple home server.

                      Originally posted by Goodsounds View Post
                      Just as you described your old machine, mine gets bogged down every few weeks and I reboot it. I don't find that to be a problem.
                      Again, I'd say non-technical users would disagree with that. Having to reboot a computer periodically to listen to music would send most non-techie people screaming back to the CD player, which never bogs down, or looking for an alternative solution that didn't periodically bog down, which is where a Linux recommendation would crop up.

                      Originally posted by Goodsounds View Post
                      Who cares if linux is free?
                      It's one of many factors, but for some it's the deciding factor, so it's worth mentioning. Personally, I'd spend some money to avoid having to struggle to get a machine working right, so I'd probably pay more for a copy of Linux than for Windows if that were the only option. Luckily for me the cheaper option also presents fewer hassles.

                      Comment


                        #12
                        Tried Linux.....

                        My squeezecenter box only runs slimcenter and a few tools, and has separate drives for data and O/S, so it was easy for me to remove the Windows XP system drive and put it a new drive on which I put Ubuntu 8.10.

                        The data drives are NTFS, and since I have nearly two TB of data, changing these to to EXT4 or NFS is not practical. Getting Ubuntu to play nice with NTFS was the first headache I had. I have some experience with Linux but I am not a Linux expert by any means, so it took hours to get this right.

                        Next, since this machine runs in a closet without a keyboard or screen, getting Ubuntu to work nicely on remote desktop through all my firewalls etc took another several hours.

                        Installing squeezecenter worked OK. But getting it to properly scan my library and see all the album art was a pain.

                        Once that was solved I found I couldn't install MuvUndercover, which is essential for maintaining the album art for my collection of obscure CDs. I tried using WINE but under Wine the application was so slow it wasn't useable. I've found no Linux app that does what MuvUndercover does. This is a show stopper. And this is one of the problems I have with Linux generally, it's a great system which doesn't support everything I need, and never will because it is such a small part of the worlwide installed base that all the applications and hardware that I need to use will not likely ever be available.... I ran into this before when I tried to use Ubuntu for a desktop system but found there was no usefull support for Creative Xfi sound cards. This is NOT an oddball or exotic sound card, it is mainstream yet there was NO support. Blame this on the driver manufacturer if you will, but whomever is to the blame the simple fact remained that Ubuntu could NOT work with my hardware- and so it has ZERO utility in my case. As far as I can tell, Xfi support is STILL missing from the current distribution.. c'mon it's been THREE YEARS since Xfi hardware became available...

                        OK, that issue aside- it doesn't apply here, this machine is a server and needs no sound card, let's proceed...

                        So, getting to how Ubuntu 8.10 behaved as a server: Serving up the Squeezecenter HTTP interface on port 9000 was faster but full directory searches were slower by about 50% than with XP, and overall server performance to my two SqueezeBoxes was noticeably slower. This may be the fault of the file system, or of transcoding from lossless WMA to FLAC, but at any rate it isn't acceptable so I took the Ubuntu system drive out and replaced it with the XP drive.

                        It was an interesting exercise, and worth doing. But Ubuntu 8.10 does not represent a change I can believe in, at least not for me..... ;-)
                        Last edited by milosz; 2008-11-04, 01:04.

                        Comment


                          #13
                          As I was thinking about responding to your thoughts (thanks), along comes Milosz, a person with some Linux experience, who throws Linux out the window.

                          But I'll offer some responses

                          Originally posted by CatBus View Post
                          To help illustrate "the other side of the fence"...

                          Under most circumstances, Windows will have software installed on it that may lead to trouble--anti-virus software for example. So with Windows, there's a higher risk of out-of-the-box failure, and having to dink around with your machine to get it working. With Linux, you have a much greater chance of having it just work without any hassles.
                          The company needs to continue to do more to make installation easier for people like me, but it wasn't a big deal. It was a few years ago. I recall having a small bump or two in the road, and perhaps making a phone call to support, but it was nothing that left an impression or a scar. No issues subsequently. Have assisted a few friends with installation, no big problems for them either.

                          Originally posted by CatBus View Post
                          One man's mantra is another man's good advice. Good advice is worth repeating.
                          Just like the Latin Mass used to be, some people are comforted by familiar sound patterns, without really knowing or understanding the meaning of the sound pattern. I was using the word mantra in this sense.

                          Originally posted by CatBus View Post
                          No doubt many others do too. However, I'd recommend replacing the hard drive on such an old machine, which is where it gets tricky for many non-techie users. Most users have never installed an operating system, and replacing a hard drive necessitates that. So if you're replacing the hard drive, then there is no difference to non-techies between Windows and Linux because they both are just as difficult to install. Windows gets its easy-to-use reputation because nobody ever installs it, and Linux gets its hard-to-use reputation because installing it is the very first thing you have to do with it. But other than that they're really not that different for running a simple home server.
                          Replace a hard drive, install an operating system? Nope, not me. If that needs to be done, I'll get someone else to do it or buy a new PC. Everything important is doubly backed up, no music or anything important on the machine, so it can die tomorrow and no problem.


                          Originally posted by CatBus View Post
                          Again, I'd say non-technical users would disagree with that. Having to reboot a computer periodically to listen to music would send most non-techie people screaming back to the CD player, which never bogs down, or looking for an alternative solution that didn't periodically bog down, which is where a Linux recommendation would crop up.
                          Maybe, just not my experience. The machine does more than just play music, and I think if I restricted the use, likely no need for reboots.

                          Originally posted by CatBus View Post
                          It's one of many factors, but for some it's the deciding factor, so it's worth mentioning. Personally, I'd spend some money to avoid having to struggle to get a machine working right, so I'd probably pay more for a copy of Linux than for Windows if that were the only option. Luckily for me the cheaper option also presents fewer hassles.
                          This discussion did get me thinking, so I asked one of my geeky friends over the weekend whether he thought I should try to bite off a Linux implementation on an old machine. He laughed, and said that having NEVER convinced anyone to do this, and having struggled with some who tried, he's stopped recomending it. He said he's come to understand that what makes it easier for him is having a career's worth of cumulative knowledge, and that for someone who doesn't bring that to the table, it's a mistake to even try.

                          He also cited the potential problem for apps and device support that some encounter.

                          Comment


                            #14
                            Flac

                            I would be shocked if your performance problems did not disappear after transcoding from WMA lossless to FLAC.

                            I agree it's probably not worth the trouble to switch platform if you have a workflow you like on Windows/Linux/whatever as there is no drastic performance difference at the OS level.

                            If you're handy with a scripting language you can see the performance difference for yourself without waiting days/weeks for the transcode to complete-- transcode one track to FLAC and copy it a few thousand times. Then copy the metadata from your WMA files (this is the hardest part) to the clones so each track's metadata appears different (audio data will obviously be the same for every track). This will give you a very good idea of the performance you can expect from SC after conversion to FLAC as the metadata will match your library. It would be pretty straightforward to do this by manipulating the file names and then using MP3Tag to map the file name info to file metadata (FLAC uses a format similar to OGG for metadata). Many others (better!) ways to do this as well.

                            Then move your WMAs out of the audiodir, move the clones into SqueezeCenter audiodir, rescan, and see if performance is what you expect while enjoying your one track.

                            If you must use WMA lossless (for other devices, preference, etc.), then it's probably cheaper/easier to keep 2 complete copies of your library and sync them (transcode from one to the other) when you add material. While it is possible to have a configuration such as your (server transcodes continuously) it sounds like it's not as good an experience as you would have using the more common formats natively supported by the SB players (e.g. FLAC for lossless, MP3/OGG/etc. for lossy).

                            YMMV

                            Comment


                              #15
                              Originally posted by Goodsounds View Post
                              Replace a hard drive, install an operating system? Nope, not me. If that needs to be done, I'll get someone else to do it or buy a new PC.
                              This is where you've clearly identified that you and Linux won't mix, unless manufacturers pre-install it. And there's nothing wrong with that, but until this point I wasn't convinced you'd have a bad time with Linux. Now I am. No more recommendations from me.

                              Originally posted by Goodsounds View Post
                              Everything important is doubly backed up, no music or anything important on the machine, so it can die tomorrow and no problem.
                              Good for you, because it probably will. Hard drives can last a very long time, but failure rates really start to climb after five years.

                              Originally posted by Goodsounds View Post
                              This discussion did get me thinking...
                              I wouldn't recommend it for average home use either, where application and hardware requirements are nebulous, changing, and likely to be Windows-only at some point. But we're talking about a home server running SqueezeCenter, where hardware/application requirements are fixed and known to be Linux compatible, and I absolutely recommend it--but not to you, for the reasons stated above. As long as we're talking about compatibility issues, I've seriously had many more problems finding Vista-compatible devices than Linux compatible devices--but none of these problems, Windows or Linux, would impact a home server.

                              Comment

                              Working...
                              X