Fw: Preferred Methods for SlimServer database access

[Jules Taplin]

Hey Dean.

Looking at the XML produced... it's not entirely surprising that it takes a
while - it's clearly doing a bunch of lookups along the way to produce
decent output.

Sadly... all I actually WANT is lists of filenames and directories, and
browsing by other criteria won't work for me either, so looks like I'll
need
a different approach. For right now - I think that's simply parsing the
filesystem myself.

Thanks anyway!


-- Jules


----- Original Message -----
> From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
> To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
> Sent: Friday, June 11, 2004 7:05 PM
> Subject: Re: [Developers] Preferred Methods for SlimServer database access
>
>
> > Hi Jules,
> >
> > Browsing the filesystem is indeed slow, especially if you have a lot of
> > items in a given directory.
> >
> > Try browsing by artist/album/genre, which uses an in-memory cache.
> >
> > On Jun 10, 2004, at 3:57 PM, Jules Taplin wrote:
> >
> > > Hey folks.
> > >
> > > I'd like to beg a little advice, if I may...
> > >
> > > I'm trying to get the portion of the Bemused Bridge that's responsible
> > > for browsing and adding tracks to playlists implemented. To this end,
> > > I need to query the contents of the directory structure that forms the
> > > SlimServer database.
> > >
> > > The CLI doesn't provide the information that I need, which has lead me
> > > onto the XML skin for the web-service. Whilst this clearly can be made
> > > to provide the information, it has a couple of problems:
> > >
> > > 1. It's dead slow.
> > > 2. Even when I get the data back, I then have to incur additional time
> > > to re-parse the XML back to something I can use.
> > >
> > > At a quick test - just grabbing the 'root' directory of my archive
> > > (which contains around 500 directories and files all in) takes 20
> > > seconds, and another second or so to parse out the XML. I think that's
> > > long enough for the bemused client to have decided the bluetooth link
> > > has fallen over, and it's certainly user-experience death on the other
> > > end. Worse than that... the Bemused client in it's default mode wants
> > > an entire directory TREE sent to it at once - the time it would take
> > > to extract that from browse.xml is enough for continents to re-form on
> > > a 20k+ track library.
> > >
> > > Equally... I don't _NEED_ most of the information that's in there...
> > > filename is quite good enough.
> > >
> > > So... have I missed something? My next approach is simply to issue a
> > > CLI 'pref audiodir', and parse the file structures directly myself -
> > > it'd be quick, after all - but it feels like cheating to me. Equally,
> > > it means that the bridge would now be tightly tied to the filesystem,
> > > and it needn't be otherwise.
> > >
> > > Have I missed something?
> > >
> > >
> > > Best Regards,
> > >
> > >
> > > Jules Taplin
> > >

[dean]

Hi Jules,

Tell us more about the application so we can possibly optimize the data
lookup...


On Jun 12, 2004, at 6:05 AM, Jules Taplin wrote:

> Hey Dean.
>
> Looking at the XML produced... it's not entirely surprising that it
> takes a
> while - it's clearly doing a bunch of lookups along the way to produce
> decent output.
>
> Sadly... all I actually WANT is lists of filenames and directories,
> and
> browsing by other criteria won't work for me either, so looks like
> I'll
> need
> a different approach. For right now - I think that's simply parsing
> the
> filesystem myself.
>
> Thanks anyway!
>
>
> -- Jules
>
>
> ----- Original Message -----
>> From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
>> To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
>> Sent: Friday, June 11, 2004 7:05 PM
>> Subject: Re: [Developers] Preferred Methods for SlimServer database
>> access
>>
>>
>>> Hi Jules,
>>>
>>> Browsing the filesystem is indeed slow, especially if you have a lot
>>> of
>>> items in a given directory.
>>>
>>> Try browsing by artist/album/genre, which uses an in-memory cache.
>>>
>>> On Jun 10, 2004, at 3:57 PM, Jules Taplin wrote:
>>>
>>>> Hey folks.
>>>>
>>>> I'd like to beg a little advice, if I may...
>>>>
>>>> I'm trying to get the portion of the Bemused Bridge that's
>>>> responsible
>>>> for browsing and adding tracks to playlists implemented. To this
>>>> end,
>>>> I need to query the contents of the directory structure that forms
>>>> the
>>>> SlimServer database.
>>>>
>>>> The CLI doesn't provide the information that I need, which has lead
>>>> me
>>>> onto the XML skin for the web-service. Whilst this clearly can be
>>>> made
>>>> to provide the information, it has a couple of problems:
>>>>
>>>> 1. It's dead slow.
>>>> 2. Even when I get the data back, I then have to incur additional
>>>> time
>>>> to re-parse the XML back to something I can use.
>>>>
>>>> At a quick test - just grabbing the 'root' directory of my archive
>>>> (which contains around 500 directories and files all in) takes 20
>>>> seconds, and another second or so to parse out the XML. I think
>>>> that's
>>>> long enough for the bemused client to have decided the bluetooth
>>>> link
>>>> has fallen over, and it's certainly user-experience death on the
>>>> other
>>>> end. Worse than that... the Bemused client in it's default mode
>>>> wants
>>>> an entire directory TREE sent to it at once - the time it would take
>>>> to extract that from browse.xml is enough for continents to re-form
>>>> on
>>>> a 20k+ track library.
>>>>
>>>> Equally... I don't _NEED_ most of the information that's in there...
>>>> filename is quite good enough.
>>>>
>>>> So... have I missed something? My next approach is simply to issue a
>>>> CLI 'pref audiodir', and parse the file structures directly myself -
>>>> it'd be quick, after all - but it feels like cheating to me.
>>>> Equally,
>>>> it means that the bridge would now be tightly tied to the
>>>> filesystem,
>>>> and it needn't be otherwise.
>>>>
>>>> Have I missed something?
>>>>
>>>>
>>>> Best Regards,
>>>>
>>>>
>>>> Jules Taplin
>>>>

[Jules Taplin]

Hi Dean.

It's probably not important enough to consider large-scale changes for...
but here goes...

I'm writing a Bridge - designed to emulate the Bemused
(http://bemused.sf.net) server which is designed to control winamp or
similar on Windows PC's via Bluetooth from mobile phones. It uses the
SlimServer CLI to communicate with SlimServer, and control Squeezeboxen
instead. The current release of the bridge is here:
http://www.mrtickle.org/bemusedbridge.html

Broadly speaking... it has three major sets of functions...

1. Obvious control - play, plause, stop, volume, skip forward and back, etc.
This is all in and working.
2. Playlist display - drag down the playlist to the phone, display it, and
let you pick tracks from it. This works, too.
3. Playlist selection & Music browsing. Basically - suck either the entire
directory tree, or a subset of it, to the phone, and allow it to select
music from there, and either directly play, or add to playlist. This isn't
in, partly 'cos it's the most complicated bit (had a bit of a go at it this
morning with direct filesystem access, and haven't got it to work yet), and
partly because the SlimServer CLI doesn't have a way to get to the
filesystems.

To bring it back... somebody on the list suggested the XML skin for database
access, and this indeed works, but it's way too detailed, and consequently
way too slow. The bemused client only cares about filenames anyway, so I'd
have to throw most of the detail away. Clearly, the Bemused client could
probably be extended to be smarter, but that's a little more work than I'm
keen to undertake at this point. Basically... all the bridge needs is to be
able to pull back filenames and directory names for a given path.


Does that make any more sense?


-- Jules

----- Original Message -----
From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
Sent: Saturday, June 12, 2004 3:04 PM
Subject: Re: [Developers] Preferred Methods for SlimServer database access


> Hi Jules,
>
> Tell us more about the application so we can possibly optimize the data
> lookup...
>
>
> On Jun 12, 2004, at 6:05 AM, Jules Taplin wrote:
>
> > Hey Dean.
> >
> > Looking at the XML produced... it's not entirely surprising that it
> > takes a
> > while - it's clearly doing a bunch of lookups along the way to produce
> > decent output.
> >
> > Sadly... all I actually WANT is lists of filenames and directories,
> > and
> > browsing by other criteria won't work for me either, so looks like
> > I'll
> > need
> > a different approach. For right now - I think that's simply parsing
> > the
> > filesystem myself.
> >
> > Thanks anyway!
> >
> >
> > -- Jules
> >
> >
> > ----- Original Message -----
> >> From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
> >> To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
> >> Sent: Friday, June 11, 2004 7:05 PM
> >> Subject: Re: [Developers] Preferred Methods for SlimServer database
> >> access
> >>
> >>
> >>> Hi Jules,
> >>>
> >>> Browsing the filesystem is indeed slow, especially if you have a lot
> >>> of
> >>> items in a given directory.
> >>>
> >>> Try browsing by artist/album/genre, which uses an in-memory cache.
> >>>
> >>> On Jun 10, 2004, at 3:57 PM, Jules Taplin wrote:
> >>>
> >>>> Hey folks.
> >>>>
> >>>> I'd like to beg a little advice, if I may...
> >>>>
> >>>> I'm trying to get the portion of the Bemused Bridge that's
> >>>> responsible
> >>>> for browsing and adding tracks to playlists implemented. To this
> >>>> end,
> >>>> I need to query the contents of the directory structure that forms
> >>>> the
> >>>> SlimServer database.
> >>>>
> >>>> The CLI doesn't provide the information that I need, which has lead
> >>>> me
> >>>> onto the XML skin for the web-service. Whilst this clearly can be
> >>>> made
> >>>> to provide the information, it has a couple of problems:
> >>>>
> >>>> 1. It's dead slow.
> >>>> 2. Even when I get the data back, I then have to incur additional
> >>>> time
> >>>> to re-parse the XML back to something I can use.
> >>>>
> >>>> At a quick test - just grabbing the 'root' directory of my archive
> >>>> (which contains around 500 directories and files all in) takes 20
> >>>> seconds, and another second or so to parse out the XML. I think
> >>>> that's
> >>>> long enough for the bemused client to have decided the bluetooth
> >>>> link
> >>>> has fallen over, and it's certainly user-experience death on the
> >>>> other
> >>>> end. Worse than that... the Bemused client in it's default mode
> >>>> wants
> >>>> an entire directory TREE sent to it at once - the time it would take
> >>>> to extract that from browse.xml is enough for continents to re-form
> >>>> on
> >>>> a 20k+ track library.
> >>>>
> >>>> Equally... I don't _NEED_ most of the information that's in there...
> >>>> filename is quite good enough.
> >>>>
> >>>> So... have I missed something? My next approach is simply to issue a
> >>>> CLI 'pref audiodir', and parse the file structures directly myself -
> >>>> it'd be quick, after all - but it feels like cheating to me.
> >>>> Equally,
> >>>> it means that the bridge would now be tightly tied to the
> >>>> filesystem,
> >>>> and it needn't be otherwise.
> >>>>
> >>>> Have I missed something?
> >>>>
> >>>>
> >>>> Best Regards,
> >>>>
> >>>>
> >>>> Jules Taplin
> >>>>

[Jules Taplin]

Hi Dean.

It's probably not important enough to consider large-scale changes for...
but here goes...

I'm writing a Bridge - designed to emulate the Bemused
(http://bemused.sf.net) server which is designed to control winamp or
similar on Windows PC's via Bluetooth from mobile phones. It uses the
SlimServer CLI to communicate with SlimServer, and control Squeezeboxen
instead. The current release of the bridge is here:
http://www.mrtickle.org/bemusedbridge.html

Broadly speaking... it has three major sets of functions...

1. Obvious control - play, plause, stop, volume, skip forward and back, etc.
This is all in and working.
2. Playlist display - drag down the playlist to the phone, display it, and
let you pick tracks from it. This works, too.
3. Playlist selection & Music browsing. Basically - suck either the entire
directory tree, or a subset of it, to the phone, and allow it to select
music from there, and either directly play, or add to playlist. This isn't
in, partly 'cos it's the most complicated bit (had a bit of a go at it this
morning with direct filesystem access, and haven't got it to work yet), and
partly because the SlimServer CLI doesn't have a way to get to the
filesystems.

To bring it back... somebody on the list suggested the XML skin for database
access, and this indeed works, but it's way too detailed, and consequently
way too slow. The bemused client only cares about filenames anyway, so I'd
have to throw most of the detail away. Clearly, the Bemused client could
probably be extended to be smarter, but that's a little more work than I'm
keen to undertake at this point. Basically... all the bridge needs is to be
able to pull back filenames and directory names for a given path.


Does that make any more sense?


-- Jules

----- Original Message -----
From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
Sent: Saturday, June 12, 2004 3:04 PM
Subject: Re: [Developers] Preferred Methods for SlimServer database access


> Hi Jules,
>
> Tell us more about the application so we can possibly optimize the data
> lookup...
>
>
> On Jun 12, 2004, at 6:05 AM, Jules Taplin wrote:
>
> > Hey Dean.
> >
> > Looking at the XML produced... it's not entirely surprising that it
> > takes a
> > while - it's clearly doing a bunch of lookups along the way to produce
> > decent output.
> >
> > Sadly... all I actually WANT is lists of filenames and directories,
> > and
> > browsing by other criteria won't work for me either, so looks like
> > I'll
> > need
> > a different approach. For right now - I think that's simply parsing
> > the
> > filesystem myself.
> >
> > Thanks anyway!
> >
> >
> > -- Jules
> >
> >
> > ----- Original Message -----
> >> From: "dean blackketter" <dean (AT) slimdevices (DOT) com>
> >> To: "Slim Devices Developers" <developers (AT) lists (DOT) slimdevices.com>
> >> Sent: Friday, June 11, 2004 7:05 PM
> >> Subject: Re: [Developers] Preferred Methods for SlimServer database
> >> access
> >>
> >>
> >>> Hi Jules,
> >>>
> >>> Browsing the filesystem is indeed slow, especially if you have a lot
> >>> of
> >>> items in a given directory.
> >>>
> >>> Try browsing by artist/album/genre, which uses an in-memory cache.
> >>>
> >>> On Jun 10, 2004, at 3:57 PM, Jules Taplin wrote:
> >>>
> >>>> Hey folks.
> >>>>
> >>>> I'd like to beg a little advice, if I may...
> >>>>
> >>>> I'm trying to get the portion of the Bemused Bridge that's
> >>>> responsible
> >>>> for browsing and adding tracks to playlists implemented. To this
> >>>> end,
> >>>> I need to query the contents of the directory structure that forms
> >>>> the
> >>>> SlimServer database.
> >>>>
> >>>> The CLI doesn't provide the information that I need, which has lead
> >>>> me
> >>>> onto the XML skin for the web-service. Whilst this clearly can be
> >>>> made
> >>>> to provide the information, it has a couple of problems:
> >>>>
> >>>> 1. It's dead slow.
> >>>> 2. Even when I get the data back, I then have to incur additional
> >>>> time
> >>>> to re-parse the XML back to something I can use.
> >>>>
> >>>> At a quick test - just grabbing the 'root' directory of my archive
> >>>> (which contains around 500 directories and files all in) takes 20
> >>>> seconds, and another second or so to parse out the XML. I think
> >>>> that's
> >>>> long enough for the bemused client to have decided the bluetooth
> >>>> link
> >>>> has fallen over, and it's certainly user-experience death on the
> >>>> other
> >>>> end. Worse than that... the Bemused client in it's default mode
> >>>> wants
> >>>> an entire directory TREE sent to it at once - the time it would take
> >>>> to extract that from browse.xml is enough for continents to re-form
> >>>> on
> >>>> a 20k+ track library.
> >>>>
> >>>> Equally... I don't _NEED_ most of the information that's in there...
> >>>> filename is quite good enough.
> >>>>
> >>>> So... have I missed something? My next approach is simply to issue a
> >>>> CLI 'pref audiodir', and parse the file structures directly myself -
> >>>> it'd be quick, after all - but it feels like cheating to me.
> >>>> Equally,
> >>>> it means that the bridge would now be tightly tied to the
> >>>> filesystem,
> >>>> and it needn't be otherwise.
> >>>>
> >>>> Have I missed something?
> >>>>
> >>>>
> >>>> Best Regards,
> >>>>
> >>>>
> >>>> Jules Taplin
> >>>>