PDA

View Full Version : Attn: Ben Klaas - This American Life Parser



malsbury
2005-10-27, 14:57
Ben,

I was trying to update the TAL_parser.pl script that you wrote last year to remove the step that we had to add to forward the stream through the local port that AlienBBC used to do the transcoding. Now Slimserver can play the true urls directly through mplayer. I think that I successfully did that, but I think that the thisamericanlife.org website might have changed its structure a little since I last used your script.

When I run it with --sort chrono_by_year the playlists do not build accurately. 1995.pls only gets a couple of shows and they are from 1996. All other years have a few oddballs from other years tossed in.

The playlist that get built work fine and the episode listed play great direclty through mplayer as planned.

If you could just take a quick look to see if I damaged the underlying code, or if it is just a matter of bringing the code in line with a change that they made to the TAL archive.

Thanks,

Tom Malsbury

I attached my updated TAL_parser.pl ,I added .txt to it so that the forums would allow it to be posted

bklaas
2005-10-29, 07:00
Hi Tom-- I'll take a look at it in the next few days...funny you should mention this script...I was just cutting all of my data over to a new server and was wondering to myself if anyone ever used those slimserver scripts...

Also, excited to hear that slimserver can play the URLs directly now. This means the playlists can be portable so I can use them elsewhere (in my case, on amarok, which if you are a linux user you should check out. I prefer it to all other audio players, including anything in the Win/Mac world)

Will update this thread after I take a look at the parsing code

#!/ben

stinkingpig
2005-10-29, 21:48
bklaas wrote:

>Also, excited to hear that slimserver can play the URLs directly now.
>This means the playlists can be portable so I can use them elsewhere
>(in my case, on amarok, which if you are a linux user you should check
>out. I prefer it to all other audio players, including anything in the
>Win/Mac world)
>
>
Amarok is very nifty, but unstable on my SuSE 9.3 laptop; I tend to try
with it first, then switch to Juk if/when it crashes.

--
Jack At Monkeynoodle Dot Org: It's A Scientific Venture!
"I spent all me tin with the ladies drinking gin
so across the Western ocean I must wander" -- trad.

bklaas
2005-10-30, 13:00
I just posted a v1.3 to here-
http://benklaas.com/slimserver/

Tom, I don't see any issue when testing the chrono_by_year sort. Not sure what was going on there. Can you give it another try with this new version and see if you are still seeing issues?

Anyway, I put in a few other changes to tidy some things up, and reinserted the alienbbc URL formatting as a command-line option (it's off by default now), just in case someone without mplayer would like to use this script.

Also, I'd love to know how to stream real via mplayer so I can actually *use* this script of mine! If anyone has any pointers on that, by all means reply on this thread.

#!/ben

malsbury
2005-10-30, 18:43
I just posted a v1.3 to here-
http://benklaas.com/slimserver/

Tom, I don't see any issue when testing the chrono_by_year sort. Not sure what was going on there. Can you give it another try with this new version and see if you are still seeing issues?
#!/ben
Glad to see that you are still hagning around the Slim Community. Thanks for updating the code so quickly, but I am still seeing the same problem. Here is a summary of what I am seeing. We will use 1995 as the example since it was a small year.

Here are the only two shows that should get added to the 1995.pls:

Quitting 12/29 Episode 7

Vacations 12/8 Episode 4

Here is the content of the 1995.pls:

[playlist]
NumberOfEntries=3
File1=http://thislife.org/ra/7.ram
Title1=Quitting (12.29.1995)
Length1=-1
File2=http://thislife.org/ra/4.ram
Title2=Vacations (12.08.1995)
Length2=-1
File3=http://thislife.org/ra/7.ram
Title3=Poultry Slam '95 (12.01.1995)
Length3=-1
Version=2

This is pretty close, it is just picking up Episode 3 (Poultry Slam '95) which shouldn't be included and assigning it the url of Episode 7.

Now here is the really odd part, and this may be a new Slimserver bug with the way it stores playlists in the new database system or it may be a problem with how the script formats the data, here is how it shows up in Slimserver 6.2.1:


Home / Browse Playlists / 1995

Name: 1995

All Songs Play Add to playlist
Quitting (03.07.1996) Play Add to playlist
Vacations (02.14.1996) Play Add to playlist
Quitting (03.07.1996) Play Add to playlist

So you can see that it is not pulling the correct title information from the playlist(It is getting the beginning of the title for the first two, but changeing the date, and completely wrong on the third). However, they do link through to the right url and play fine.

If you have any ideas, I'd be intersted in hearing them. You can confirm that you are not seeing this behaviour on your system? I am seeing similar errors in all years generated.

--Tom Malsbury

malsbury
2005-10-30, 19:54
Also, I'd love to know how to stream real via mplayer so I can actually *use* this script of mine! If anyone has any pointers on that, by all means reply on this thread.The missing piece for me was that I had never installed the Codecs required. I don't remember what flavor of linux you are running, but I was able to find a RedHat based RPM for my CentOS 4.1 system (w32codec-1.0-9.2.el4.mok.i386.rpm) and it finally worked.

If you think this may be your problem to you can take a look here to see if there is a package for your distro: http://rpm.pbone.net/index.php3?stat=3&limit=1&srodzaj=2&dl=40&search=w32codec&dist[]=44&dist[]=41&dist[]=37&dist[]=31&dist[]=32&dist[]=1&dist[]=28&dist[]=2&dist[]=3&dist[]=4&dist[]=5&dist[]=7&dist[]=42&dist[]=36&dist[]=35&dist[]=11&dist[]=12&dist[]=34&dist[]=13&dist[]=14&dist[]=15&dist[]=17&dist[]=18&dist[]=33&dist[]=19&dist[]=20&dist[]=23&dist[]=24&dist[]=25&dist[]=26&dist[]=27&dist[]=29&dist[]=30&dist[]=38&dist[]=39&dist[]=40&dist[]=43

I tested it according the the last step I found in this post:http://forums.slimdevices.com/showpost.php?p=57215&postcount=1


* To test that the player works, run the following :
mplayer rtsp://rmlivev8.bbc.net.uk/farm/*/ev7/live24/radio1/live/r1_dsat_g2.ra
* You should see it connect and then start to decode; once it prints out
'Starting playback', you know all's well.Hope that points you in the right direction.

--Tom Malsbury

bklaas
2005-10-31, 13:48
Actually, I think the script is golden.

On the problems you see in the 1995.pls file:

There is an error in the html on http://www.thislife.org/pages/archives/archivemain.html. If you go there and hold your mouse over the "NA" by the 1995 entry "Poultry Slam" you will see what the problem is. I think that's the only error of that type on the page, so I'm choosing to ignore it rather than kludge a workaround.

as for the title/playlist thing...that's interesting. Looks a lot like this bug that I opened ages ago, but should have been fixed in 6.x slimserver:
http://bugs.slimdevices.com/show_bug.cgi?id=735

I've reproduced what you're seeing there, Tom. Probably this bug needs to get reopened.

malsbury
2005-10-31, 14:14
Actually, I think the script is golden.
... I'm choosing to ignore it rather than kludge a workaround.I see what the problem I was seeing is now... I guess I never really looked at the code to see exactly what page you were scrapping. I always thought that you were searching the individual year archive pages:

http://www.thislife.org/pages/archives/archive95.html
http://www.thislife.org/pages/archives/archive96.html
http://www.thislife.org/pages/archives/archive97.html
http://www.thislife.org/pages/archives/archive98.html etc..

These pages only list new shows for each year without rebroadcast of older episodes. When I looked at the playlists I was seeing episodes out of numeric order in each playlists. Now that I see what is going on and where you are getting the info I agree the code is golden.


As for the title/playlist thing...that's interesting. Looks a lot like this bug that I opened ages ago, but should have been fixed in 6.x slimserver:
http://bugs.slimdevices.com/show_bug.cgi?id=735

I've reproduced what you're seeing there, Tom. Probably this bug needs to get reopened.I will definitly open a bug for this, I just installed the 10/31 nightly and now it is not listing the titles at all, just the URL is listed as the title. The old playlists used to show up fine in 6.1.2, then mislabled in 6.2.1 10/27, then not labeled at all in 6.2.1 10/31.

Thanks againg for your quick attention. Is mplayer working for you yet?

--Tom Malsbury

bklaas
2005-10-31, 14:54
sounds like there is some active work going on with the playlist logic. If you open a bug, make sure to reference bug 735 as related, as it appears it is...

no love on mplayer, but probably serves me right...I just jumped the Fedora bandwagon and have started using Gentoo. I'm fairly pleased with it, but get totally stuck when I hit errors like this...so off the slimserver forums and on to the gentoo forums i go...

mediumspicy ~ # emerge mplayer
Calculating dependencies ...done!
>>> emerge (1 of 2) media-video/realplayer-10.0.6 to /
>>> Downloading https://helixcommunity.org/download.php/1589/RealPlayer-10.0.6.776-20050915.i586.rpm
--15:51:02-- https://helixcommunity.org/download.php/1589/RealPlayer-10.0.6.776-20050915.i586.rpm
=> `/usr/portage/distfiles/RealPlayer-10.0.6.776-20050915.i586.rpm'
Resolving helixcommunity.org... 207.188.25.135
Connecting to helixcommunity.org|207.188.25.135|:443... connected.
ERROR: Certificate verification error for helixcommunity.org: unable to get local issuer certificate
To connect to helixcommunity.org insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
!!! Couldn't download RealPlayer-10.0.6.776-20050915.i586.rpm. Aborting.
mediumspicy ~ #

bklaas
2005-10-31, 15:52
alright, I got mplayer working (thanks to the gentoo forums), now it's on to making slimserver work with these real audio streams. Tom, have you gotten that far? I'm assuming there needs to be a new line in convert.conf to handle .ra streams?

And while we're at it, mplayer should be able to handle a few more...wma, asx, etc. Any pointers there? http://www.pfarrell.com/music/slimserver/tips.html has some good pointers on wma over ffmpeg, but I'd like to know the ropes with mplayer...

malsbury
2005-10-31, 17:16
alright, I got mplayer working (thanks to the gentoo forums), now it's on to making slimserver work with these real audio streams. Tom, have you gotten that far? I'm assuming there needs to be a new line in convert.conf to handle .ra streams?

And while we're at it, mplayer should be able to handle a few more...wma, asx, etc. Any pointers there? http://www.pfarrell.com/music/slimserver/tips.html has some good pointers on wma over ffmpeg, but I'd like to know the ropes with mplayer...That's great to here you got it up and running. Now, if you install the new AlienBBC 0.99 for linux, it sets up a seperate slimserver.conf that overides any seperate entries in the convert.conf. It adds the handles for both RealAudio and WindowsMedia streaming that you will need. It also takes care of wma local files so you no longer need to use ffmpeg.

Eventhough I don't directly use the AlienBBC plugin, it is useful just to have it installed so it takes care of all the *.conf stuff that always takes so much tweaking. The only change that I made was to change the volume=0 value to volume=8 on both the WMA and RA settings since they were a little quiter then my AAC (.m4a) conversions using FAAD. If you need to make a similar change, you will see where to make it based on the nice #comment sections in all the AlienBBC files.

I'm assuming that you have installed the _pre7 version of mplayer, if Gentoo only is up to the _pre6 release, the settings are in the files too, you just have to remove the comment marks.

Any other questions, feel free to ask. It has made all the difference for me once I got it working. I've even updated the PublicRadioFanBrowser plugin to take advantage of streaming WM and RA. See my post in the Plugins Forum.

--Tom Malsbury

bklaas
2005-11-01, 08:11
From reading some other threads on slimforums, it looks like convert.conf is for files only, not streams. So, I'm quite baffled as to how I can stream real audio out slimserver via mplayer (without AlienBBC, as Malsbury mentions in the first post to this thread)...I'm assuming *something* needs to be configured *somewhere*.

bklaas
2005-11-01, 08:24
whoops, missed your post there Tom. Will try your recommendation on installing AlienBBC and see how that goes. Thanks for your input!
#!/ben

bklaas
2005-11-01, 10:49
never did get mplayer/real to work...to be honest, I'm not surprised. There is nothing I've flailed with more through my time with slimserver than trying to listen to Alien streams...

opened up a separate thread on it...
http://forums.slimdevices.com/showthread.php?t=17732

Looks like things are getting better, but I just can't seem to cover the last mile to having this work. oh well.

bklaas
2005-11-12, 20:17
TAL_parser.pl v1.4 is at http://benklaas.com/slimserver. I put in a couple of lines to avoid duplicate streams in the playlists (i.e., dates where reruns were aired)

FWIW, I did finally get mplayer to pipe real audio to slimserver. The key was to make sure that mplayer was built to use the win32codecs realaudio support instead of RealPlayer's (go figure). This HOWTO is what got me through it (it's gentoo-specific, but lots of good info there)
http://gentoo-wiki.com/HOWTO_Rip_Streams_With_MPlayer

Also Tom, thanks for the pointer on turning the volume up for the real streams...worked like a charm.

cheers,
#!/ben