PDA

View Full Version : slimserver is a memory hog



Kevin Walsh
2003-11-30, 11:52
Roy M. Silvernail [roy (AT) rant-central (DOT) com]:
> > >
> > > SlimServer is a bit of a memory hog and we do plan on improving this
> > > in future updates. But even so, 60MB is more than is expected for
> > > 15000 files. What operating system are you using?
> > >
> > That's pretty close to what I'm seeing - 12,001 songs and 42.9MB under
> > RH9.
> >
> Just for reference: 3,265 songs, 25MB, Gentoo 1.4 with 5.0.1
>
For reference:

Albums: 313
Tracks: 5,140
Artists: 2,380
Genres: 32

Memory: 28MB
O/S: GNU/Linux Red Hat 8.0
Server: 5.0.1

I'm happy with the memory usage, btw. If I could afford enough CDs to
create 15,000 track files then I'd be even happier. :-)

--
_/ _/ _/_/_/_/ _/ _/ _/_/_/ _/ _/
_/_/_/ _/_/ _/ _/ _/ _/_/ _/ K e v i n W a l s h
_/ _/ _/ _/ _/ _/ _/ _/_/ kevin (AT) cursor (DOT) biz
_/ _/ _/_/_/_/ _/ _/_/_/ _/ _/

Victor Brilon
2003-11-30, 19:05
This is fun :)

Tracks: 8265
Albums: 929

Memory: 30.5 MBs
OS: RH 9
Server: 5.0

Kevin Walsh wrote:
> Roy M. Silvernail [roy (AT) rant-central (DOT) com]:
>
>>>>SlimServer is a bit of a memory hog and we do plan on improving this
>>>>in future updates. But even so, 60MB is more than is expected for
>>>>15000 files. What operating system are you using?
>>>>
>>>
>>>That's pretty close to what I'm seeing - 12,001 songs and 42.9MB under
>>>RH9.
>>>
>>
>>Just for reference: 3,265 songs, 25MB, Gentoo 1.4 with 5.0.1
>>
>
> For reference:
>
> Albums: 313
> Tracks: 5,140
> Artists: 2,380
> Genres: 32
>
> Memory: 28MB
> O/S: GNU/Linux Red Hat 8.0
> Server: 5.0.1
>
> I'm happy with the memory usage, btw. If I could afford enough CDs to
> create 15,000 track files then I'd be even happier. :-)
>

David Ranch
2003-11-30, 20:05
I understand that the goal of the Slim server is to be fast and scalable but I
think this thread is pointing one thing out. This software is going to require
a LOT of RAM to scale for LOTS of media. The question is.. should the memory
footprint really be that large?

Couldn't the data structure be compressed within RAM? I realize this will
increase the overall latency of the system but for some people, this will be
acceptable. Maybe have the option to have the system keep the uncompressed
data structure in RAM for 30-60 minutes. If the Slim server isn't used within
that time, compress the structure and release the memory back. Once the server
is used again, uncompress it and go through the decision tree again.

Moving to the future, has it been considered to move the music database into an
ACTUAL database like mySQL or PostGres? It's this exact reason why Apple went
with iTunes.


Tracks: 7,608
Albums: 1,026

Memory: 34.7MB
OS: Mandrake 7.0 (heavily patched)
Slim: 5.0.1CVS


PPs. Is there any way to get this info from the web interface? I could only
fine this from the remote via: Settings --> information --> library stats


--David


>Tracks: 8265
>Albums: 929
>
>Memory: 30.5 MBs
>OS: RH 9
>Server: 5.0
>
>Kevin Walsh wrote:
>> Roy M. Silvernail [roy (AT) rant-central (DOT) com]:
>>
>>>>>SlimServer is a bit of a memory hog and we do plan on improving this
>>>>>in future updates. But even so, 60MB is more than is expected for
>>>>>15000 files. What operating system are you using?
>>>>>
>>>>
>>>>That's pretty close to what I'm seeing - 12,001 songs and 42.9MB under
>>>>RH9.
>>>>
>>>
>>>Just for reference: 3,265 songs, 25MB, Gentoo 1.4 with 5.0.1
>>>
>>
>> For reference:
>>
>> Albums: 313
>> Tracks: 5,140
>> Artists: 2,380
>> Genres: 32
>>
>> Memory: 28MB
>> O/S: GNU/Linux Red Hat 8.0
>> Server: 5.0.1
>>
>> I'm happy with the memory usage, btw. If I could afford enough CDs to
>> create 15,000 track files then I'd be even happier. :-)
>>
>

Jack Coates
2003-11-30, 22:59
On Sun, 2003-11-30 at 19:05, David Ranch wrote:
> I understand that the goal of the Slim server is to be fast and scalable but I
> think this thread is pointing one thing out. This software is going to require
> a LOT of RAM to scale for LOTS of media. The question is.. should the memory
> footprint really be that large?

Define lots. These numbers don't look impossibly inordinate to me in a
scaling sense, though smaller would of course be better.

>
> Couldn't the data structure be compressed within RAM? I realize this will
> increase the overall latency of the system but for some people, this will be
> acceptable. Maybe have the option to have the system keep the uncompressed
> data structure in RAM for 30-60 minutes. If the Slim server isn't used within
> that time, compress the structure and release the memory back. Once the server
> is used again, uncompress it and go through the decision tree again.
>
> Moving to the future, has it been considered to move the music database into an
> ACTUAL database like mySQL or PostGres?

yes, it has. Keep in mind that said database needs to be transparent to
the user and easy to install on three OS's, which rules out Postgresql.
BDB is more like it, but licensing costs could be an issue. I'll tell ya
one thing, Postgresql or MySQL will chew up even more memory to manage
this information, though either would probably be tons faster.

> It's this exact reason why Apple went
> with iTunes.
>
>
> Tracks: 7,608
> Albums: 1,026
>
> Memory: 34.7MB
> OS: Mandrake 7.0 (heavily patched)
> Slim: 5.0.1CVS
>
>
> PPs. Is there any way to get this info from the web interface? I could only
> fine this from the remote via: Settings --> information --> library stats
>

Try sudo cat /proc/`pidof slimserver`/status
....
--
Jack at Monkeynoodle Dot Org: It's A Scientific Venture...

"I'm celebrating my love for you with a pint of beer and a new tattoo,
and if you haven't noticed yet, I'm more impressionable when my cement
is wet."
-- Greetings To The New Brunette from Talking Poetry With The Taxman by
Billy Bragg

Andreas Jung
2003-11-30, 23:28
--On Sonntag, 30. November 2003 21:59 Uhr -0800 Jack Coates
<jack (AT) monkeynoodle (DOT) org> wrote:

>
> yes, it has. Keep in mind that said database needs to be transparent to
> the user and easy to install on three OS's, which rules out Postgresql.
> BDB is more like it, but licensing costs could be an issue. I'll tell ya
> one thing, Postgresql or MySQL will chew up even more memory to manage
> this information, though either would probably be tons faster.

I don't care so much about the memory finger print but about the
implications.
When I search for a title or something else I often see that the slimserver
stands still no music comes out of the Slimp3 until the search is finished.
Although raising the priority of the process did not solve the problem.
Means
there is something in the slimserver architecture that leads to drop out
(maybe swap-in,swap-out on unixes machines might be responsible). I have
a reasonable fast machine (2400XP, 512MB RAM) and such drop-outs
are not very much acceptable. I would have solved the problem on my own
but hacking Perl code is not much fun because I am a Python programmer :-)

-aj

David Cantrell
2003-12-01, 03:30
On Sun, Nov 30, 2003 at 07:05:11PM -0800, David Ranch wrote:
>I understand that the goal of the Slim server is to be fast and scalable but I
>think this thread is pointing one thing out. This software is going to require
>a LOT of RAM to scale for LOTS of media. The question is.. should the memory
>footprint really be that large?
>
>Couldn't the data structure be compressed within RAM?

Broadly speaking, perl doesn't release memory back to the OS when it
goes out of scope (ie when the reference count reaches zero) and perl
provides no malloc/free mechanism for the programmer to control memory
(de)allocation. Instead, memory is kept hanging around and may be re-used
later by the same perl process. Or it might not be re-used. In either
case it will eventually get swapped out when the OS notices that it's
not being used. The perl process appearing to be using a huge chunk of
RAM doesn't really matter as the OS should do the right thing (all bets
are off on Windows, of course!).

[insert muttering about mixed data and instruction memory and page
sizes]

One of perl's design goals is speed, at the expense of memory.

> increase the overall latency of the system but for some people, this will be
> acceptable. Maybe have the option to have the system keep the uncompressed
> data structure in RAM for 30-60 minutes. If the Slim server isn't used within
> that time, compress the structure and release the memory back.

Not possible with perl unless you write your own custom data types and
memory allocation voodoo in C/XS - and that's insane. That's almost all
the hard work involved in writing a C application, so you might as well
write the whole damned thing in C. And then it's bye-bye easy plugins,
and bye-bye feature upgrades cos the programmers are spending all their
time hunting for memory allocation bugs :-)

Using tied hashes to store data in dbm files may help, although I
haven't read the code to see if that would be suitable.

--
David Cantrell | http://www.cantrell.org.uk/david

All principles of gravity are negated by fear
-- Cartoon Law V