PDA

View Full Version : large-scale server solution ??



moschous
2008-01-16, 09:29
Hi, i have already put this post on the General Discussion forum but i didnt get any clear answer. Maybe you could be more helpfull.

http://forums.slimdevices.com/showthread.php?t=41937

Our aim is to develop a music-service where slimdevices will be able to connect to and get music. We intend to release the service in Greece and provide Greek music on clients (slim devices).

The service will come up with a web-portal where users will be able to buy content, create their own profiles, create playlists, share playlists, chat for music, etc..

The service will offer both mp3 files on-demand and playlists (targeting multiple mp3 files at once). We dont intend to broadcast any live streaming station, but we will offer lets say virtual radio stations (e.g. playlists targeting many different categories rok, R&B, etc..).

Now, which are the problems we can see on this:

1. We need somehow to integrate our web application (where users browse on a big mp3-library, buy content, create private or public playlists, etc) with the slimserver/squeezeCenter or any slimproto-based server we eventually use.
We need to have different users, to access different playlists and different audio libraries.

What do i understand is that we need a custom slimproto-based
to communicate with our client's squeezeboxes.
For example, lets see a possible usage-scenario: The owner of a specific SB will browse on web portal, buy content and create its own playlist. We need somehow to force its own squeeze box to start playing (tune in) on that specific playlist. We need to do that through our web portal and not through slimserver's web interface.

2. We need somehow to identify the different devices everytime they connect to the service. Is that possible? I saw that while SB connects to Squeezenetwork it sends a pin code. Is that available on slimserver?

3. And the most difficult problem is the issue "scalability".

But before going on this i want to make sth clear. There are two different steps/levels while SB gets music.

a. The first one is the communication level (slimproto protocol) where a slimproto - based server tells SB what to play, where to connect, etc

b. The second step/level is the delivery of real data (e.g. mp3 files). In case where we deliver local files/playlists on slim server the slim server serves those files. In case where we choose something that is remotely hosted then the slim server just notify the SB(via slimproto) which in turns connects to the remote area to get the content (e.g. icecast, podcasts, shoutcasts or even a pure web server).

is that right?

So, even if we are going to use the slimserver for the front part (the slimproto-communication) we need to find a way to scale the second level/step which is the delivery of audio files. Of course the second level will be the most heavy in terms of traffic, etc... (think about for more than 500 SB's)

So, based on the matters i noted above what is the best choice i have?

What are you recommending on me?

radish
2008-01-16, 10:03
1. We need somehow to integrate our web application (where users browse on a big mp3-library, buy content, create private or public playlists, etc) with the slimserver/squeezeCenter or any slimproto-based server we eventually use.
We need to have different users, to access different playlists and different audio libraries.

What do i understand is that we need a custom slimproto-based
to communicate with our client's squeezeboxes.
For example, lets see a possible usage-scenario: The owner of a specific SB will browse on web portal, buy content and create its own playlist. We need somehow to force its own squeeze box to start playing (tune in) on that specific playlist. We need to do that through our web portal and not through slimserver's web interface.

Assuming you build your own server you can of course add all the features you need such as multiple libraries, etc. Once a player is connected to your service you can also control it completely from the server - no problems. The issue is going to be getting the client connected to your server. The SB firmware knows how to do 3 things to get a connection:

1) Scan the local subnet for a server
2) Connect to a server outside the local subnet given a specific IP
3) Connect to SqueezeNetwork

The only one of these which will work for you is (2), which will require your users to enter the IP address for your server. Not ideal.



2. We need somehow to identify the different devices everytime they connect to the service. Is that possible? I saw that while SB connects to Squeezenetwork it sends a pin code. Is that available on slimserver?

It's nothing to do with slimserver, you need the client (SB) to send something unique to the service when it connects. I'm not 100% sure, but I don't think that currently happens on a regular slimproto connection - the PIN is something added specifically for SN.



3. And the most difficult problem is the issue "scalability".

Actually I think it's one of the easier problems :)



a. The first one is the communication level (slimproto protocol) where a slimproto - based server tells SB what to play, where to connect, etc

It's more involved than that. Slimproto controls everything the player does, including handling IR commands and drawing the screen. The server connection must be persistent at all times.



b. The second step/level is the delivery of real data (e.g. mp3 files). In case where we deliver local files/playlists on slim server the slim server serves those files. In case where we choose something that is remotely hosted then the slim server just notify the SB(via slimproto) which in turns connects to the remote area to get the content (e.g. icecast, podcasts, shoutcasts or even a pure web server).

All the player knows how to do is get an audio stream from the server (using slimproto) or do basic HTTP streaming. You'll need to support one or the other.



So, even if we are going to use the slimserver for the front part (the slimproto-communication) we need to find a way to scale the second level/step which is the delivery of audio files. Of course the second level will be the most heavy in terms of traffic, etc... (think about for more than 500 SB's)

Setup a bunch of HTTP stream servers and a bunch of slimproto servers. When the client hits play, bounce them out to a load-balanced URL for the HTTP stream. That will get you a "personal radio" style experience. If you want full browsing, skip, forward, rewind, etc then you'll need to serve the audio via slimproto as well. Regardless, you'll just need a bunch of servers and some load balancing. The real issue is keeping things responsive.



So, based on the matters i noted above what is the best choice i have?
What are you recommending on me?
I recommend you build a third party service and negotiate with Logitech to have it added to SqueezeNetwork - like Pandora/Rhapsody/etc. It would be FAR easier than building all this stuff yourself and give your users a better experience. You'll still need your own infrastructure for serving of course, but a lot of your real problems would be taken care of.

stinkingpig
2008-01-16, 10:16
On Jan 16, 2008 8:29 AM, moschous
<moschous.33aevz1200501001 (AT) no-mx (DOT) forums.slimdevices.com> wrote:
>
> Hi, i have already put this post on the General Discussion forum but i
> didnt get any clear answer. Maybe you could be more helpfull.
> ...

If I were you, I'd email slimdevices directly and ask to be put in
touch with business development. You're wanting to build an equivalent
to Pandora/Slacker/Last.fm, which is already integrated into the
ecosystem through Squeezenetwork; it will be much simpler in the long
run to do just that instead of reimplementing your own Squeezenetwork,
IMHO.
--
"I spent all me tin with the ladies drinking gin,
So across the Western ocean I must wander" -- traditional

moschous
2008-01-16, 12:34
It's more involved than that. Slimproto controls everything the player does, including handling IR commands and drawing the screen. The server connection must be persistent at all times.


So, even the SB connects to a remote HTTP server or any remote internet radio what is drawn on the SB's screen is controlled by slim server? (please correct me if i am wrong)



All the player knows how to do is get an audio stream from the server (using slimproto) or do basic HTTP streaming. You'll need to support one or the other.


Setup a bunch of HTTP stream servers and a bunch of slimproto servers. When the client hits play, bounce them out to a load-balanced URL for the HTTP stream. That will get you a "personal radio" style experience. If you want full browsing, skip, forward, rewind, etc then you'll need to serve the audio via slimproto as well. Regardless, you'll just need a bunch of servers and some load balancing. The real issue is keeping things responsive.


Could you please make more clear what do you mean by saying HTTP streaming?

I can imagine:

1. a pure HTTP web server for audio on demand (playlists, etc)
2. icecast with ices/ices2

.. any other alternative?

Have you ever heard anybody on using any other well-known streaming server like QTSS or windows media services, etc...?

finally, by using any other delivering solution other than slim server is not possible to control the stream, right(skip, forward, rewind, etc..)?



I recommend you build a third party service and negotiate with Logitech to have it added to SqueezeNetwork - like Pandora/Rhapsody/etc. It would be FAR easier than building all this stuff yourself and give your users a better experience. You'll still need your own infrastructure for serving of course, but a lot of your real problems would be taken care of.

where should i

radish
2008-01-16, 14:04
So, even the SB connects to a remote HTTP server or any remote internet radio what is drawn on the SB's screen is controlled by slim server? (please correct me if i am wrong)

In that case I _think_ the text is being drawn locally (any more experienced devs please feel free to correct me) - because the SB is doing the remote connection and so it has the ICE headers. However, as soon as you use the remote to do anything (browse menus, volume, etc) it will hit the server.



Could you please make more clear what do you mean by saying HTTP streaming?

I can imagine:

1. a pure HTTP web server for audio on demand (playlists, etc)
2. icecast with ices/ices2

I was thinking of option 2, icecast, as that's what most radio stations use. Podcasts are played by the same mechanism (AFAIK) so it must be able to handle simple files (option 1) as well. No idea if it can handle pulling down playlists and parsing those.



Have you ever heard anybody on using any other well-known streaming server like QTSS or windows media services, etc...?

No. But I'm not an expert in streaming technologies - if the server looks like a simple HTTP server and the format it sends is understood by the SB (mp3/ogg/wmv/wav/flac) it should work. But for the real details you'll need to either find a friendly firmware dev or test it yourself :)



finally, by using any other delivering solution other than slim server is not possible to control the stream, right(skip, forward, rewind, etc..)?

You can pause and skip back/forwards in the playlist, but that's it. If the SB can't load a remote playlist (and I don't know whether it can) then you're stuck with a single playlist item streaming the whole list, in which case skip obviously won't do anything useful.

The key thing to remember is the old company name - Slim Devices. The player is exceptionally dumb, all the interesting stuff happens in the server. They've recently been adding more functionality to the firmware but it's still very limited (and likely to remain so).

moschous
2008-01-17, 02:17
In that case I _think_ the text is being drawn locally (any more experienced devs please feel free to correct me) - because the SB is doing the remote connection and so it has the ICE headers. However, as soon as you use the remote to do anything (browse menus, volume, etc) it will hit the server.


just one clarification. Do you mean that even in case where i change the volume/brightness in SB, it sends the command to the slimserver and then slimserver sends back the command to the SB in order to make the action?

If yes, isnt it fool to happen?

radish
2008-01-17, 06:59
In general all the processing is done on the server. In most cases the SB doesn't even understand the IR signals itself, it sends them back to the server which translates them and figures out what to do.

The advantage of this, other than reduced complexity on the client, is ultimate customization. All this behaviour is controlled by the server so it's all hackable and configurable without changing the firmware - which is closed source. For an example look at the IR blaster plugin which allows the SB to act as an IR repeater for practically any device, or the MusicInfo plugin which allows complex customisation of the Now Playing and Volume displays. None of this would be possible with a more traditional heavy-client model.