PDA

View Full Version : Integrating the holdings of the Internet Archive with my PC'sholdings to play on squeeze?



Brewster Kahle
2004-01-07, 22:55
I work with the Internet Archive and a recent buyer of the squeezebox
(very cool).

What I would like to figure out is how to get the Archive's music
holdings to be available to all squeezebox users (including myself).

we have limited dev folks, but we have lots of music and plenty of
bandwidth.

We have over 7000 concert recordings from 400 bands. These are
currently in shn and flac formats, but most of these will be offered in
mp3 in the coming months.
We have over 1000 "netlabel" ep's and albums-- free mp3s and oggs
curated by volunteer labels. mostly electronica and idm. I am
warming to this stuff as background.

We have Open Archives Initiatives (OAI) xml interfaces, rss feeds,
m3u's for netlabel albums, and we can produce other things.

If anyone has ideas of how we can, or better yet, others, can help us
list our music in such a way that squeezeboxers and others can
integrate this public library into your private libraries, that would
be great. then when slimdevices comes out with the video equivalent
we will be all set to offer our movies as well.

Thanks for squeezebox. we want to help this open world win!

-brewster

digital librarian
internet archive

Bill Fenner
2004-01-08, 00:15
On Jan 7, 2004, at 9:55 PM, Brewster Kahle wrote:
> What I would like to figure out is how to get the Archive's music
> holdings to be available to all squeezebox users (including myself).

As a start, I downloaded http://www.archive.org/stream/8bp038 to my
playlists directory on the server, renamed it with an .m3u extension,
and was able to access it from the playlists menu on the squeezebox.

Bill

kdf
2004-01-08, 00:56
Quoting Brewster Kahle <brewster (AT) archive (DOT) org>:

>
> I work with the Internet Archive and a recent buyer of the squeezebox
> (very cool).
>
> What I would like to figure out is how to get the Archive's music
> holdings to be available to all squeezebox users (including myself).

Perhaps serve up the files using womething like shoutcast so that the META
information can be grabbed by the slimp3. If you make the playlists browseable
via RSS or XML is should be easy enough to create a plugin to navigate through
the lists and play a chosen stream.

Take a look at the shoutcast plugin here:
http://www.dur.ac.uk/p.j.heslin/amusements/shoutcast_plugin.html

oh, and be sure to join the developers list for slimdevices so that this
discussion can continue in more details as they come up.

cheers,
kdf

Caleb Epstein
2004-01-08, 07:08
On Wed, Jan 07, 2004 at 11:15:01PM -0800, Bill Fenner wrote:

> On Jan 7, 2004, at 9:55 PM, Brewster Kahle wrote:
>
> >What I would like to figure out is how to get the Archive's music
> >holdings to be available to all squeezebox users (including myself).
>
> As a start, I downloaded http://www.archive.org/stream/8bp038 to my
> playlists directory on the server, renamed it with an .m3u extension,
> and was able to access it from the playlists menu on the squeezebox.

And where did that URL come from? I'm doing my best to comb
the Live Music Archive
(http://www.archive.org/audio/etree.php), and I'm finding
nothing about streaming audio.

Brewster:

To have the LMA available for on-demand streaming would be
fantastic! I'm doubtful that you could stream the lossless
files over the Internet - the bandwidth demands would be too
great (~172 Kbytes/sec per stream) - but you could provide
lossy streams of every show by either transcoding the files to
MP3 on-demand or by archiving an MP3 copy of every show
alongside its lossless counterpart and serving up those files
on-request.

The latter idea was proposed at some point, but was IIRC
vehemently opposed by many of the tapers who supply the files
that the LMA hosts. It seems like a complete non-starter for
that reason.

The former idea would require considerable CPU horsepower to
pull off - I know that my home server can transcode
WAV to MP3 in about 4x real-time. Perhaps with a dedicated
transcoding "farm" of machines or dedicated hardware MP3
encoders you could serve up a large number of these streams.

--
Caleb Epstein | bklyn . org | Before borrowing money from a friend, decide
cae at | Brooklyn Dust | which you need more.
bklyn dot org | Bunny Mfg. | -- Addison H. Hallock

dean
2004-01-08, 08:43
Hi Brewster,

This is very exciting! I'd love to help build a plug-in that provides
direct access to the Internet Archive content and ship it with
SlimServer by default.

Let's take a look at the content and figure out the best way to browse
it from Squeezebox. Folks, post your ideas and let's get going!

-dean



On Jan 7, 2004, at 9:55 PM, Brewster Kahle wrote:

>
> I work with the Internet Archive and a recent buyer of the squeezebox
> (very cool).
>
> What I would like to figure out is how to get the Archive's music
> holdings to be available to all squeezebox users (including myself).
>
> we have limited dev folks, but we have lots of music and plenty of
> bandwidth.
>
> We have over 7000 concert recordings from 400 bands. These are
> currently in shn and flac formats, but most of these will be offered
> in mp3 in the coming months.
> We have over 1000 "netlabel" ep's and albums-- free mp3s and oggs
> curated by volunteer labels. mostly electronica and idm. I am
> warming to this stuff as background.
>
> We have Open Archives Initiatives (OAI) xml interfaces, rss feeds,
> m3u's for netlabel albums, and we can produce other things.
>
> If anyone has ideas of how we can, or better yet, others, can help us
> list our music in such a way that squeezeboxers and others can
> integrate this public library into your private libraries, that would
> be great. then when slimdevices comes out with the video equivalent
> we will be all set to offer our movies as well.
>
> Thanks for squeezebox. we want to help this open world win!
>
> -brewster
>
> digital librarian
> internet archive
>
>
>
>
>

Jonathan Greene
2004-01-08, 08:47
This would rock - pun intended. Would this be limited to Squeezebox
only? How about us original guys?


On Jan 8, 2004, at 10:43 AM, dean blackketter wrote:

> Hi Brewster,
>
> This is very exciting! I'd love to help build a plug-in that provides
> direct access to the Internet Archive content and ship it with
> SlimServer by default.
>
> Let's take a look at the content and figure out the best way to browse
> it from Squeezebox. Folks, post your ideas and let's get going!
>
> -dean
>
>
>
> On Jan 7, 2004, at 9:55 PM, Brewster Kahle wrote:
>
>>
>> I work with the Internet Archive and a recent buyer of the squeezebox
>> (very cool).
>>
>> What I would like to figure out is how to get the Archive's music
>> holdings to be available to all squeezebox users (including myself).
>>
>> we have limited dev folks, but we have lots of music and plenty of
>> bandwidth.
>>
>> We have over 7000 concert recordings from 400 bands. These are
>> currently in shn and flac formats, but most of these will be offered
>> in mp3 in the coming months.
>> We have over 1000 "netlabel" ep's and albums-- free mp3s and oggs
>> curated by volunteer labels. mostly electronica and idm. I am
>> warming to this stuff as background.
>>
>> We have Open Archives Initiatives (OAI) xml interfaces, rss feeds,
>> m3u's for netlabel albums, and we can produce other things.
>>
>> If anyone has ideas of how we can, or better yet, others, can help us
>> list our music in such a way that squeezeboxers and others can
>> integrate this public library into your private libraries, that would
>> be great. then when slimdevices comes out with the video
>> equivalent we will be all set to offer our movies as well.
>>
>> Thanks for squeezebox. we want to help this open world win!
>>
>> -brewster
>>
>> digital librarian
>> internet archive

dean
2004-01-08, 09:12
Sorry, SLIMP3 users are always welcome...


On Jan 8, 2004, at 7:47 AM, Jonathan Greene wrote:

> This would rock - pun intended. Would this be limited to Squeezebox
> only? How about us original guys?
>
>
> On Jan 8, 2004, at 10:43 AM, dean blackketter wrote:
>
>> Hi Brewster,
>>
>> This is very exciting! I'd love to help build a plug-in that
>> provides direct access to the Internet Archive content and ship it
>> with SlimServer by default.
>>
>> Let's take a look at the content and figure out the best way to
>> browse it from Squeezebox. Folks, post your ideas and let's get
>> going!
>>
>> -dean
>>
>>
>>
>> On Jan 7, 2004, at 9:55 PM, Brewster Kahle wrote:
>>
>>>
>>> I work with the Internet Archive and a recent buyer of the
>>> squeezebox (very cool).
>>>
>>> What I would like to figure out is how to get the Archive's music
>>> holdings to be available to all squeezebox users (including myself).
>>>
>>> we have limited dev folks, but we have lots of music and plenty of
>>> bandwidth.
>>>
>>> We have over 7000 concert recordings from 400 bands. These are
>>> currently in shn and flac formats, but most of these will be offered
>>> in mp3 in the coming months.
>>> We have over 1000 "netlabel" ep's and albums-- free mp3s and oggs
>>> curated by volunteer labels. mostly electronica and idm. I am
>>> warming to this stuff as background.
>>>
>>> We have Open Archives Initiatives (OAI) xml interfaces, rss feeds,
>>> m3u's for netlabel albums, and we can produce other things.
>>>
>>> If anyone has ideas of how we can, or better yet, others, can help
>>> us list our music in such a way that squeezeboxers and others can
>>> integrate this public library into your private libraries, that
>>> would be great. then when slimdevices comes out with the video
>>> equivalent we will be all set to offer our movies as well.
>>>
>>> Thanks for squeezebox. we want to help this open world win!
>>>
>>> -brewster
>>>
>>> digital librarian
>>> internet archive
>
>

jacobdp
2004-01-08, 19:48
On Thu, 8 Jan 2004 07:43:52 -0800, you wrote:
>Let's take a look at the content and figure out the best way to browse
>it from Squeezebox. Folks, post your ideas and let's get going!

OK, here's an idea: Web Menus.

(I haven't actually tried implementing this, so I'm not sure how
feasable it would be...)

A Web Menu is a placeholder file that contains the URL of a playlist,
to be fetched via HTTP. Think "symbolic link", but handled by
SlimServer and pointing to something on the Web. The playlist is then
used as a menu.

So, Joe Sqeezebox-Owner puts "InternetArchive.wpl" in his music
directory. It contains something like this:

[Archive]
displayname=Internet Archive music library
url=http://www.archive.org/audio/slimdevices/root.pls

When it is navigated to, via the regular browse interface, SlimServer
automagically opens up an HTTP connection and fetches root.pls. (That
filename is not displayed to the user). The contents of root.pls are
then shown as the menu. I hope I'm being clear here...

This is distinguished from a regular pls or m3u by the fact that it
contains only one item by definition, and the redirect is automatic.

I just tested and SlimServer supports nested playlists, although the
display is a bit messed up (top line reads "foo1.pls/0" rather than
"foo1.pls/Nested Playlist"), but that shouldn't be too hard to fix.
Also, IIRC Apache could be set up to generate these "menus"
automatically.

The actual MP3s would then be served up via whatever's convenient,
probably just HTTP streams.


Does this sound like it will work? Comments? Advice? Flames?

Ideally, this would allow the extention of the SlimServer music
library _and menu system_ directly to sites on the 'net. All we need
is a good marketing name... :-)

- Jacob
--
"Terminak #3 has bad keyboard. Pkease fix."

dean
2004-01-11, 09:20
At one point in the past, this actually worked. Remote playlists were
fetched and browsable via HTTP. I don't think anybody actually used
it, so I don't know if it fell into disrepair or not, but in theory it
should work.

If it doesn't work now, I'd bet that it wouldn't be hard to get going
again.

-dean

On Jan 8, 2004, at 6:48 PM, Jacob Potter wrote:

> On Thu, 8 Jan 2004 07:43:52 -0800, you wrote:
>> Let's take a look at the content and figure out the best way to browse
>> it from Squeezebox. Folks, post your ideas and let's get going!
>
> OK, here's an idea: Web Menus.
>
> (I haven't actually tried implementing this, so I'm not sure how
> feasable it would be...)
>
> A Web Menu is a placeholder file that contains the URL of a playlist,
> to be fetched via HTTP. Think "symbolic link", but handled by
> SlimServer and pointing to something on the Web. The playlist is then
> used as a menu.
>
> So, Joe Sqeezebox-Owner puts "InternetArchive.wpl" in his music
> directory. It contains something like this:
>
> [Archive]
> displayname=Internet Archive music library
> url=http://www.archive.org/audio/slimdevices/root.pls
>
> When it is navigated to, via the regular browse interface, SlimServer
> automagically opens up an HTTP connection and fetches root.pls. (That
> filename is not displayed to the user). The contents of root.pls are
> then shown as the menu. I hope I'm being clear here...
>
> This is distinguished from a regular pls or m3u by the fact that it
> contains only one item by definition, and the redirect is automatic.
>
> I just tested and SlimServer supports nested playlists, although the
> display is a bit messed up (top line reads "foo1.pls/0" rather than
> "foo1.pls/Nested Playlist"), but that shouldn't be too hard to fix.
> Also, IIRC Apache could be set up to generate these "menus"
> automatically.
>
> The actual MP3s would then be served up via whatever's convenient,
> probably just HTTP streams.
>
>
> Does this sound like it will work? Comments? Advice? Flames?
>
> Ideally, this would allow the extention of the SlimServer music
> library _and menu system_ directly to sites on the 'net. All we need
> is a good marketing name... :-)
>
> - Jacob
> --
> "Terminak #3 has bad keyboard. Pkease fix."
>
>
>

jacobdp
2004-01-11, 10:25
On Sun, 11 Jan 2004 08:20:00 -0800, you wrote:
>At one point in the past, this actually worked. Remote playlists were
>fetched and browsable via HTTP. I don't think anybody actually used
>it, so I don't know if it fell into disrepair or not, but in theory it
>should work.

Yep.

I did a bit more testing, and it does work, sort of. There are a few
issues:

1. A link to a remote playlist has the music-note icon in the corner,
rather than the arrow. I assume this has something to do with the
isFOO functions, right? Shouldn't be too hard to fix.

2. The first line display is messed up. Rather than giving the "title"
of that playlist (I'm not sure what the Right Thing for that is), it
gives the numerical index from the previous playlists. This happens to
all playlists within other playlists. I looked through the code, and
can't find what causes this. Any idea?

3. Fairly minor, but... there's no "connecting to server" indication
when going into a remote playlist, just a fairly noticable delay.

4. I'm using plain ol' Apache to stream, and at the end of the song,
it stops playing but doesn't advance - the timer keeps counting up and
it still says "playing", though. I haven't played with proper
streaming radio, so I doubt this will be an issue in practice.

It would be cool to get this working... I'm also playing with some PHP
for auto-generating the playlists, but no luck yet.

- Jacob

--
"Terminak #3 has bad keyboard. Pkease fix."

dean
2004-01-11, 10:33
Just about what I suspected. Let's take this to the developer list and
fix all these problems. Should be pretty quick.

-dean

On Jan 11, 2004, at 9:25 AM, Jacob Potter wrote:

> On Sun, 11 Jan 2004 08:20:00 -0800, you wrote:
>> At one point in the past, this actually worked. Remote playlists were
>> fetched and browsable via HTTP. I don't think anybody actually used
>> it, so I don't know if it fell into disrepair or not, but in theory it
>> should work.
>
> Yep.
>
> I did a bit more testing, and it does work, sort of. There are a few
> issues:
>
> 1. A link to a remote playlist has the music-note icon in the corner,
> rather than the arrow. I assume this has something to do with the
> isFOO functions, right? Shouldn't be too hard to fix.
>
> 2. The first line display is messed up. Rather than giving the "title"
> of that playlist (I'm not sure what the Right Thing for that is), it
> gives the numerical index from the previous playlists. This happens to
> all playlists within other playlists. I looked through the code, and
> can't find what causes this. Any idea?
>
> 3. Fairly minor, but... there's no "connecting to server" indication
> when going into a remote playlist, just a fairly noticable delay.
>
> 4. I'm using plain ol' Apache to stream, and at the end of the song,
> it stops playing but doesn't advance - the timer keeps counting up and
> it still says "playing", though. I haven't played with proper
> streaming radio, so I doubt this will be an issue in practice.
>
> It would be cool to get this working... I'm also playing with some PHP
> for auto-generating the playlists, but no luck yet.
>
> - Jacob
>
> --
> "Terminak #3 has bad keyboard. Pkease fix."
>
>
>