Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 7 123 ... LastLast
Results 1 to 10 of 63
  1. #1
    Senior Member Michaelwagner's Avatar
    Join Date
    Apr 2005
    Location
    Oakville, Ontario, Canada
    Posts
    2,024

    Slimserver footprint, time and space, and an architectural suggestion

    I'm not sure the developers forum is the right place for this, but ... no other place seemed more suitable, so ... here goes.

    I'm trying to understand why the slimserver has such a large footprint in time and space. By space I mean RAM, by time I mostly mean scan time.

    I can read tags for my set of about 8500 songs in about 2 minutes. That's using FOXPRO (an interpreted language, like perl) on a 1.8GHz Pentium with 500MB ram. I did some experiments in a moderately compiled language (Visual Basic 6, the last one before .net). It was 2 orders of magnitude faster. So 12 seconds would be a good expectation for a commercial product.

    The memory footprint for the Foxpro compiled code is about 55MB including disk caching, but disk caching seems to be about 40MB of it, so I'm guessing the program footprint itself is about 15MB. As a database program, it builds rapid lookup indexes and everything in that time and space. After it's finished building, I can do keystroke by keystroke interactive song lookup when I'm searching for that song whose title contains the phrase "s'wonderful" or "robin" or whatever.

    The machine where slimserver runs is a 600MHz machine with 128MB RAM. It's so slow when it scans the tags, I can have not just a coffee but a shower and a shave while it's doing it. At a third the speed, I would expect about 6 minutes if it's all done in perl, at least an order of magnitude less with some optimization.

    The memory footprint seems to be basically everything available on the 128MB machine (just to head off the question, each machine has it's own copy of the song database on a disk local to the machine, so there's no networking differences here).

    Slim doesn't report the scan time, so I have accurate numbers, just a feeling that it's slightly longer than forever.

    My direct motivation for asking is, I admit, a somewhat unusual application (I DJ with the squeezebox), but the issues raised seem to touch on other peoples experience too, as I found when I tried a search for the keywords "slow startup".

    It seems to me the slimserver has two fundamentally different tasks to perform - feeding data to the hardware and managing a user interface.

    As it turns out, I don't need the user interface stuff, the database, etc and don't use it (except for debugging & testing). But that's just me.

    When I'm DJing, I shut off the rebranded web browser (I think this is the slimserver process).

    But can I shut off the whole user interface? The song database, the scanning, everything?

    Clearly, the server has to scan the tags of the song it's about to play, in order to put the name up on the screen, to answer CLI calls for what song are we playing, etc. Does it in fact do it that way, or does it go to the database?

    And if it goes to the database currently, can that be changed?

    It seems to me that the current slim server code should really be 2 modules: hardware driver and user interface.

    Feeding the hardware is time critical - if you don't feed it fast enough, you get dropouts.

    If they are on the same machine (the "vanilla" installation), they should contend for operating system resources at different priorities and communicate over an interface.

    One could also chose to run them on different machines. The hardware driver on a NAS box like the buffalo or linksys, close to the disk data, and the user interface, with it's larger resource needs but less time critical, on some other box "closer" to the user.

    In my searching, I found a number of people who wrote about something in the user interface part of the code stopping the hardware interface. And I've done this myself. So it is a concern.

    In summary, my questions:

    1. Why is tag scanning so resource intensive?

    2. I thought the switch to a database system in 6 was supposed to address this problem ... my experience was that it was worse (I switched back to 5, for other reasons, so I can't check at the moment).

    3. Given that it is so resource intensive (and potentially, even if you improved it, it's a large problem ... for every improvement in code efficiency, someone out there will have a larger library of songs ... ) how about these solutions:

    4. Can the scanning and database service be turned off in the current code (something like scan never, flush cache)? Does anything I care about (song information like title and duration) break?

    5. Can the user interface be separated from the "hardware feeding"
    a. in the current code?
    b. as a future enhancement?

    Thanks in advance for your time and considered opinions.

    Michael

  2. #2

    a thought

    One possible (but not necessarily recommended) way to solve your problem would be to write a special purpose server. Since the protocols are openly documented, you (or someone) could probably write a server in a compiled language that might have a very
    small footprint that could do exactly what you want.

    --dan

  3. #3
    Senior Member Michaelwagner's Avatar
    Join Date
    Apr 2005
    Location
    Oakville, Ontario, Canada
    Posts
    2,024
    True. But I think the perl code is fast enough for driving the hardware, and there are tremendous advantages to sticking to the mainstream codebase.

    Thanks for the offer, though.

    Michael

  4. #4
    Robin Bowes
    Guest

    Slimserver footprint, time and space,and an architectural suggestion

    Michaelwagner wrote:

    > It seems to me the slimserver has two fundamentally different tasks to
    > perform - feeding data to the hardware and managing a user interface.


    Exactly right.

    I've been saying (at every available opportunity) that audio streaming
    should be broken out into a separate process/thread so it can be
    prioritised by the O/S.

    I've looked into doing this myself a few times but the slimserver code
    is pretty monolithic and it's not a task I feel I'm up to.

    What's required is to remove the modules that deal with audio streaming
    from slimserver.pl and start them up in a separate process, and decide
    on a mechanism for the audio streaming process and main slimserver
    process to communicate. It wouldn't require a particularly complex
    protocol - all the audio streaming has to do is to send audio files from
    a playlist to a player until it runs out. It should also communicate
    status back to the slimserver process for the player display updates,
    but this must be a non-blocking operation.

    One other possibility is that a stand-alone streaming process could be
    written to support upnp and thus open up streaming to the SB from
    Windows Media Connect, etc.

    I remain hopeful that someone will bite the bullet one of these days...

    R.
    --
    http://robinbowes.com


  5. #5
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by Michaelwagner
    5. Can the user interface be separated from the "hardware feeding"
    a. in the current code?
    b. as a future enhancement?
    This is something I've been wondering as well. It would seem to be a lot more flexible having a lightweight service / daemon / device-driver that just drives the Squeezebox hardware and doesn't require the HTTP server and database server to function. Having all that stuff loaded into memory just to use an interface like the CLI doesn't make a lot of sense.
    ________
    smoke weed every day
    Last edited by JJZolx; 2011-01-22 at 16:40.

  6. #6
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by Michaelwagner
    The machine where slimserver runs is a 600MHz machine with 128MB RAM. It's so slow when it scans the tags, I can have not just a coffee but a shower and a shave while it's doing it. At a third the speed, I would expect about 6 minutes if it's all done in perl, at least an order of magnitude less with some optimization.
    BTW, it sounds like you're running a Windows system. Which version of Windows are you running in 128MB of RAM? You can run XP in 256MB, but I'd consider 512MB to be the minimum unless performance is a secondary consideration. Running XP in 128MB is nothing short of painfully slow.
    ________
    teen vid
    Last edited by JJZolx; 2011-01-22 at 16:40.

  7. #7
    Senior Member Michaelwagner's Avatar
    Join Date
    Apr 2005
    Location
    Oakville, Ontario, Canada
    Posts
    2,024
    Quote Originally Posted by JJZolx
    Which version of Windows are you running in 128MB of RAM?
    W98SE

    (BTW, on my first attempt, that's all I wrote, and the forum software told me my reply was too short and I had to write at least 10 characters. Really? Why?)

  8. #8
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by Michaelwagner
    W98SE
    Doable in 128MB, but with SlimServer taking up 50-75MB, not very comfortable. I understand, though, that was one of the original points of this thread. :-)

    I'm curious - How do you use the Squeezebox for DJing? Given that you have to run a computer to function as a server to drive the Squeezebox, why not use something like a USB DAC (such as the M-Audio Audiophile USB) and then eliminate SlimServer? You'd have the added flexibility of using different software packages to control the music - foobar, Winamp, iTunes, etc. The SlimServer/Squeezebox combo is handy for a lot of scenarios, but I'm not seeing how DJing is one of them.
    ________
    BMW K75C
    Last edited by JJZolx; 2011-01-22 at 16:41.

  9. #9
    Senior Member Michaelwagner's Avatar
    Join Date
    Apr 2005
    Location
    Oakville, Ontario, Canada
    Posts
    2,024
    Quote Originally Posted by JJZolx
    I'm curious - How do you use the Squeezebox for DJing?
    It's fairly straightforward. I have a database of songs, criteria like speed, suitableness for dancing, what style of dance goes with the song, etc, and I select the next few songs I want to play and it watches them play out and sends the next one over.

    It was important that I maintain the queue and not the MP3 player, because I sometimes change my mind (that is to say, the floor changes, the people change, and what was a suitable song 2 songs ago is no longer suitable) and I reserve the right to change my mind about the next song until about 10 seconds before the end of the current one.

    Given that you have to run a computer to function as a server to drive the Squeezebox, why not use something like a USB DAC (such as the M-Audio Audiophile USB) and then eliminate SlimServer?
    Several reasons:
    1. I started with an Audiotron, and the Audiotron did all of the byte fetching itself, in response to (relatively) high level commands like "play this next song".
    2. When I had to abandon the Audiotron (it was heavy, and it was discontinued), I looked around for something similar, ironically, because I didn't want to write the byte dribbling level of software. I assumed that when I bought the slim package, I was getting someone else's code that already did this.

    You'd have the added flexibility of using different software packages to control the music - foobar, Winamp, iTunes, etc.
    I'm not seeing an advantage there. I don't generally switch CD heads in the middle of a dance, and I don't want to switch to a different software package either. I wrote my software because nothing I saw out there (when I first started the package, 2 years ago ... situation may be different now) did what I as a DJ want. Sure, all sorts of packages do what people do at house parties, but not real stuff.

    And no, I don't do beat mixing (hate it, actually). The style of DJing I do believes that the artist put the tip and the tail of the song there for a reason and they should be left there. Can you imagine reading a series of books without reading the first and last chapters, just the stuff in between?

    The SlimServer/Squeezebox combo is handy for a lot of scenarios, but I'm not seeing how DJing is one of them.
    Well, I was hoping for a "slimmer" slim server, but even now, it's not bad. My app. is written in high level terms (queue this song now) and I leave the bytes to someone else.

    DJing is a hobby. I do it to earn enough money to pay for my CDs (I like paying for my music - it keeps blues and blues artists alive). I don't particularly have time to write low level hardware driving software, and didn't much want to.

    Looks like I might have to anyways. Or at least learn how to prune a little perl :-)

    Michael

  10. #10
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by Michaelwagner
    It's fairly straightforward. I have a database of songs, criteria like speed, suitableness for dancing, what style of dance goes with the song, etc, and I select the next few songs I want to play and it watches them play out and sends the next one over.

    I don't particularly have time to write low level hardware driving software, and didn't much want to.
    I see. You have your own custom application that needs a simple means to play music tracks that you queue. Currently it uses the CLI to tell SlimServer which files to feed a Squeezebox.

    So what you really need is an interface similar to the CLI to talk to some piece of playback software and then you could use a computer with either an internal or external sound card. There must be more than a couple ways to do this.

    For one idea, take a look at foobar2000 and the available command line parameters to foobar2000.exe. You have things like /add /play /pause. And it's under 4MB in memory. Your program would shell out and issue a command such as:

    "C:\Program Files\foobar2000\foobar2000.exe" /play "D:\Music\Cal Tjader\Soul Sauce\01 Soul Sauce.mp3"
    ________
    70
    Last edited by JJZolx; 2011-01-22 at 16:42.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •