PDA

View Full Version : SS7.0 Framework for Parsing and Displaying Remote Information Sources



Triode
2007-08-17, 19:28
Given 7.0 is getting close to beta and hopefully plugin authors are beginning to look at it, I though I'd post a plugin I have been working to which exploits some additional features in 7.0.

The purpose of the plugin is to fetch, parse and display information from arbitary web sources on the player, SS or jive/cli. An opml file defines a menu of remote urls to fetch and if approprite a perl file to use to parse them into a format suitable for displaying. This means simple HTML::Parser scripts can be used to convert web pages into information displays. The server also includes cookie support, so it is possible to log into a web site and parse its page for display on a player (or jive application). [The same approach is used by the 7.0 version of Alien, in this case producing station lists rather than information displays.]

The plugin includes a very simple framework for adding additional parsers - these are added by creating a folder within the pluging folder and adding an opml file for the new menu and parser file.

Here's the current version, together with a simple jive applet which allows jive to read the information. Both should be considered as work in progress (especially the jive one as the api keeps changing!) However I'm hoping it provokes interest of other plugin developers to develop parsers for their favorite information sources.

Triode
2007-08-19, 06:08
I've created a wiki page for this plugin and added a couple of addons to demonstrate how these are writen:

http://wiki.slimdevices.com/index.cgi?InformationBrowser

Here's what you could be seeing on the jive emulator... (Same text is available on the player interface and SS web) Please have a look and consider adding addons for your favourite web sites..

erland
2007-08-19, 11:02
Two questions:

1.
Won't you get into problem with the PAR packaging in 7.0 ?
I had some issues when I tried to convert Custom Scan plugin to 7.0. The problem was that my code contained "use $module" inside the plugin code to load additional modules which was placed in a separate directory. The result is that the PAR packaging utility didn't realize that this *.pm files was needed and didn't include them in the PAR file. To solution for me was to add statically "use xxx" directive in the main plugin file, which basically makes it impossible for users to add their own module without repackaging the PAR.
If you have found a solution to this, I'm very interested to know how you did it.

2.
If I understand this correctly there will be a "Information Browser" menu which contains sub element for all opml files. The urls in the opml files is static and there is no way of adding dynamic parameters to them ?
The reason for these questions is that I started to think that it might be useful to have a "Information Browser" menu in the track details menu. For example if you right click on a song in the "Now Playing" menu. To make this useful you would have to be able to pass the song title or album title of the currently selected track as the url. As an example this could be used to make a module that lookup information on LastFM related to the selected song.
I'm guessing that this might be a bit out of the scope for this plugin, or would it be possible to enhance it this way ?

Triode
2007-08-19, 11:27
> 1.
> Won't you get into problem with the PAR packaging in 7.0 ?
> I had some issues when I tried to convert Custom Scan plugin to 7.0.
> The problem was that my code contained "use $module" inside the plugin
> code to load additional modules which was placed in a separate
> directory. The result is that the PAR packaging utility didn't realize
> that this *.pm files was needed and didn't include them in the PAR
> file. To solution for me was to add statically "use xxx" directive in
> the main plugin file, which basically makes it impossible for users to
> add their own module without repackaging the PAR.
> If you have found a solution to this, I'm very interested to know how
> you did it.

Yes thats something that needs to be worked though. The server still
supports plugins not packaged as PARs (the default ones are done this way).
I tend to think this may be appropriate for some plugins.. In the case of
this plugin, the use to load the parser is actually in the server's xml code
so as long as the add on parsers are added as non PAR files they should be
loaded dynamically.

> If I understand this correctly there will be a "Information Browser"
> menu which contains sub element for all opml files. The urls in the
> opml files is static and there is no way of adding dynamic parameters
> to them ?

Don't need to be - the url for one menu level is created by the parser for
the previous menu level. There is the ability to reduce the default cache
time of 5 minutes to allow urls to be refetched. If you want to dynamically
build the content then it would be necessary to turn off this caching.

> The reason for these questions is that I started to think that it might
> be useful to have a "Information Browser" menu in the track details
> menu. For example if you right click on a song in the "Now Playing"
> menu. To make this useful you would have to be able to pass the song
> title or album title of the currently selected track as the url. As an
> example this could be used to make a module that lookup information on
> LastFM related to the selected song.
> I'm guessing that this might be a bit out of the scope for this plugin,
> or would it be possible to enhance it this way ?

That requires Slim::Buttons::TrackInfo to add extra options to call your
plugin (which would then be able to call InfoBrowser with a url for the
track). You may be able to register a protocol handler for type file to do
that, but I've not tried...

erland
2007-08-19, 11:54
> The reason for these questions is that I started to think that it might
> be useful to have a "Information Browser" menu in the track details
> menu. For example if you right click on a song in the "Now Playing"
> menu. To make this useful you would have to be able to pass the song
> title or album title of the currently selected track as the url. As an
> example this could be used to make a module that lookup information on
> LastFM related to the selected song.
> I'm guessing that this might be a bit out of the scope for this plugin,
> or would it be possible to enhance it this way ?

That requires Slim::Buttons::TrackInfo to add extra options to call your
plugin (which would then be able to call InfoBrowser with a url for the
track). You may be able to register a protocol handler for type file to do
that, but I've not tried...

I replace the standard TrackInfo in the Custom Browse plugin with a version that makes it possible to add new sub menus. So adding the menu under "Now Playing" isn't any problem if you have Custom Browse installed.

I think what I like is actually to be able to use the parser and specify an url for it to use dynamically. If I understand this correctly the InfoBrowser plugin is just the main menu and all the sub menus is implemented in xmlbrowser and the parsers.
You initiate xmlbrowser by passing it an opml file and in the opml file you specify the url and also a parser which will parse the result retrieved when calling the url. The parser will parse the result and create additional sub menus based on the parsed result. Is this correct ?

If this is the case, I think everything would be solved if I was able to specify the url in the call to xmlbrowser and it would use that url instead of the one passed in the opml file. It this already possible or will xmlbrowser currently always read the root url from the opml file ?

Triode
2007-08-19, 12:09
> I replace the standard TrackInfo in the Custom Browse plugin with a
> version that makes it possible to add new sub menus. So adding the menu
> under "Now Playing" isn't any problem if you have Custom Browse
> installed.
>
> I think what I like is actually to be able to use the parser and
> specify an url for it to use dynamically. If I understand this
> correctly the InfoBrowser plugin is just the main menu and all the sub
> menus is implemented in xmlbrowser and the parsers.
> You initiate xmlbrowser by passing it an opml file and in the opml file
> you specify the url and also a parser which will parse the result
> retrieved when calling the url. The parser will parse the result and
> create additional sub menus based on the parsed result. Is this correct
> ?

Something like the following should probably work from your plugin... You
can make the url whatever you want and this should be able to include params
to send to the server.

Turn on formats.xml debugging if it doesn't quite work and see what it is
doing. Note, by default it will cache the parsed answer for 5 mins.

my %params = (
'url' => $url,
'title' => $title,
'parser' => 'Some::Perl::File,
);

Slim::Buttons::Common::pushMode( $client, 'xmlbrowser', \%params );

erland
2007-08-19, 12:17
Note, by default it will cache the parsed answer for 5 mins.Is there some way to turn this off, for example by sending it as a parameter in the %params hash ?

Is the cache per url ?
As an example if I initiated xmlbrowser with different request parameter values in the url for different tracks, would this result in that I get a cached copy for each track ?

Triode
2007-08-19, 12:48
> Triode;221655 Wrote:
>> Note, by default it will cache the parsed answer for 5 mins.Is there some
>> way to turn this off, for example by sending it as a
> parameter in the %params hash ?
>
> Is the cache per url ?
> As an example if I initiated xmlbrowser with different request
> parameter values in the url for different tracks, would this result in
> that I get a cached copy for each track ?
>

It's per url so I think you should be ok. If not set 'cachetime' in the
toplevel hash in your parser to a lower value - say 30 seconds.

You will see what is going on with the formats.xml debug.

To clear the cache: rm -Rf Cache/FileCache

This should only remove cache and nothing else of more imporance stored in
the Cache dir!

bpa
2007-09-03, 15:47
I've ported my AlienBBC "BBC What's On" addon over to Infobrowser. A zip of the addon is attached.

One minor issues - I create a level of menu in the opml file for Days of the week but I have had to add an URL to keep the XMLBrowser happy even though the parser does not use the URL.

Is there a neater way ?

Triode
2007-09-04, 11:46
Bryan,

Looks good - very useful.

Re the dummy URL: The way it currently works is to:
1) fetch the content for the remote url
2) run the perl parser on it
3) return to xmlbrowser to display the next level of menus

So for your application, I was assuming that your top level list of station
names to urls would be done in the opml file, not in the perl file - guess
this means it gets quite long in this case? So your file
BBCWhatsOnDayParser.pm could just be in the opml file rather than a perl
file as the menu is not dynamic?

----- Original Message -----
From: "bpa" <bpa.2wcwhb1188859801 (AT) no-mx (DOT) forums.slimdevices.com>
To: <developers (AT) lists (DOT) slimdevices.com>
Sent: Monday, September 03, 2007 11:47 PM
Subject: Re: [Developers] SS7.0 Framework for Parsing and Displaying
RemoteInformation Sources


>
> I've ported my AlienBBC "BBC What's On" addon over to Infobrowser. A
> zip of the addon is attached.
>
> One minor issues - I create a level of menu in the opml file for Days
> of the week but I have had to add an URL to keep the XMLBrowser happy
> even though the parser does not use the URL.
>
> Is there a neater way ?
>
>
> +-------------------------------------------------------------------+
> |Filename: InfoBrowserAddons.ZIP |
> |Download: http://forums.slimdevices.com/attachment.php?attachmentid=3258|
> +-------------------------------------------------------------------+
>
> --
> bpa
> ------------------------------------------------------------------------
> bpa's Profile: http://forums.slimdevices.com/member.php?userid=1806
> View this thread: http://forums.slimdevices.com/showthread.php?t=37693
>
>