Slimproto change proposal

**philippe_44** · 2020-11-15, 02:02

Slimproto change proposal

I was thinking about the options to inform a player that the URL to be queried is HTTPS instead of HTTP, per discussion here https://github.com/ralph-irving/squeezelite/issues/108.

Today, slimproto does not tell the player if it should try HTTPS instead of HTTP for direct streaming, although squeezelite (and friends) now support SSL and can inform LMS with the canHTTPS option in "HELO" message. It was not an issue as long as SSL only happens on port 443 in which case squeezelite would try SSL based on the port number sent in "strm_s message". Unfortunately, more and more servers are using SSL on other ports, so we have made workaround to support that and try secured / non-secured but it's a bit cumbersome and has some security flaws.

Initially, I thought that there was unused bit in some strm_s fields and we could re-use them without breaking compatibility, but I was too optimistic and it seems that none are "really" available. It's also difficult to extend the strm_s message because the string of headers to be sent to the server immediately follow the strm_s header of 24 bytes, so LMS can't offset that string without being sure that the player is fine with that. Similarly, an updated player does not know if LMS has moved it or not, we have the usual complex backward/forward compatibility issue.

So I wanted to submit here a proposal for two options (keep in mind that no SB device supports SSL, only squeezelite - and friends

- does)

Code:

# Squeezebox control for tcp stream
#
#u8_t command;// [1]'s' = start, 'p' = pause, 'u' = unpause, 'q' = stop, 'f' = flush
#u8_t autostart;// [1]'0' = don't auto-start, '1' = auto-start, '2' = direct streaming, '3' direct streaming + autostart
#u8_t mode;// [1]'m' = mpeg bitstream, 'p' = PCM
#u8_t pcm_sample_size;// [1]'0' = 8, '1' = 16, '2' = 24, '3' = 32
#u8_t pcm_sample_rate;// [1]'0' = 11kHz, '1' = 22, '2' = 32, '3' = 44.1, '4' = 48, '5' = 8, '6' = 12, '7' = 16, '8' = 24, '9' = 96
#u8_t pcm_channels;// [1]'1' = mono, '2' = stereo
#u8_t pcm_endianness;// [1]'0' = big, '1' = little
#u8_t threshold;// [1]Kb of input buffer data before we autostart or notify the server of buffer fullness
#u8_t spdif_enable;// [1]  '0' = auto, '1' = on, '2' = off
#u8_t transition_period;// [1]seconds over which transition should happen
#u8_t transition_type;// [1]'0' = none, '1' = crossfade, '2' = fade in, '3' = fade out, '4' fade in & fade out, '5' = crossfade-immediate
#u8_t flags;// [1]
#               //      0x80 - loop infinitely
#               //      0x40 - stream without restarting decoder
#               //      0x20 - Rtmp (SqueezePlay only)
#               //      0x10 - SqueezePlay direct protocol handler - pass direct to SqueezePlay
#               //      0x08 - output only right channel as mono
#               //      0x04 - output only left channel as mono
#               //      0x02 - polarity inversion right
#               //      0x01 - polarity inversion left
#
#u8_t output_threshold`;// [1]Amount of output buffer data before playback starts in tenths of second.
#u8_t slaves;// [1]number of proxy stream connections to serve
#u32_t replay_gain;// [4]replay gain in 16.16 fixed point, 0 means none
#u16_t server_port;// [2]server's port
#u32_t server_ip;// [4]server's IP
#// ____
#// [24]
#
#[HTTP headers to be sent to the server start here]

1- Shoehorn

In strm_s, the flags field has a "RTMP" option which I think is not really used as I don't see many services using RTMP these days, but I'm not really sure. My understanding is that this flag was added and only supported by the Radio and/or the Touch. My reading of squeezelite code is that it does not support RTMP (and does not set "canDecodeRtmp" flag in HELO)

So, we have a exclusive usage here and we could share that bit. If the player supports HTTPS, then LMS can reliably set that flag to indicate that HTTPS is needed even without knowing precisely the model of the player. When it is needed for RTMP (maybe through a plugin) then it's not for squeezelite.

Any "old" squeezelite version will work as it ignores that RTMP/SSL bit and it will continue to do (for a while ...) the multiple SSL/non-SSL attempts. Any update version of squeezelite (or player that claims to support HTTPS) will look at that bit and open an SSL connection immediately. With time, we could deprecate the squeezelite "funky" code or we could detect LMS version and not use that code. It assumes that no future player will do RTMP *and* SSL (which seems a reasonable assumption)

This is quick, simple, very located in LMS code but does not provide much future proofness (?)

NB: there might be an alternative option which is to use the "slaves" byte which seems to be used by rhapsody only and I don't now exactly for what. So we would have exactly the same exclusive situation wrt to player supporting both rhapsody and SSL

2- Extend slimproto

In that case, we want to make room in strm_s message's header for future extension, so we need to move (offset) the HTTP headers string. We cannot just put something after the header because squeezelite (and maybe other players) use the hardcoded length of the strm_s header to calculate the length of HTTP headers string.

So first we would have to let LMS know that players support slimproto extension, which could be done with a "slimproto=XXX" field in "HELO" message. Then, when that flag is set, LMS could offset the header string. I suggest we make that offset a variable length rather than hardcode it. Bear in mind as well that a new squeezelite version, although it has sent "slimproto=2" in it's HELO capabilities, does not know what the LMS server will send in strm_s.

So LMS, when it has detected a player that supports "extended" slimproto, would add at the end of the 24 bytes of strm_s

Code:

#u8_t marker = 0;// [1]must be NULL so that a squeezelite client knows this is not headers string
#u8_t flags;// [1]to be defined, but at least it contains SSL flag
#u16_t offset; // [1]      number of bytes to add to find headers string, no including the 2 previous bytes
#u8_t bytes // [offset]         [... some optional bytes ...]
#[ HTTP headers ]

The benefit of that variable offset is to make room for further LMS/strm_s changes without breaking compatibility with existing squeezelite/players version. The "marker" byte allows any squeezelite version to know that this is not the start of headers string. We could consider other fields as well like the version of slimproto/strm_s with which LMS is answering.

Another benefit of "offset" is that we could add a "can('extendSTRMs')" method for player-type plugins that want to add their own private bits to a strm_s message and we would add retrieve these by calling the extendSTRMs method

This second option requires much more changes to LMS (it's not horrific, but less straightforward)

So ... let me know if we should move that discussion to github or not and please bear in mind that I made that up as I was writing it, so please be benevolent for any embarrassing logic mistake I might have made

**bpa** · 2020-11-15, 13:23

I can provide some background to the RTMP implementation and hopefully help in assessing whether it needs to be retained.

The RTMP implementation is for Flash audio streaming support and not Apple RTMP. It was implemented by Triode who implemented AlienBBC and BBCiPlayer to play BBC radio programs.

Triode was UK based and around 2005-2010 there was a interest in playing BBC streams as BBC used to use RealAudio but was moving services (TV & Radio) to Flash streaming.

BBCiPlayer plugin was written to replace AlienBBC and it implemented RTMP in Perl to decode a Flash Audio stream and produce ADTS AAC for the SB players. However this meant playing BBC Radio On-demand and quality live streams required an LMS server.
Around the same time Touch and Radio were released and Triode was heavily involved in the Squeezeplay development. To support playing BBC Radio Live and On-demand BBC using mysqeezebox.com (i.e. no local LMS), Triode wrote a BBC Radio Applet and also re-implemented RTMP protocol support (in C and not LUA I think) as part of Touch/Radio firmware. The "spdr://" protocol handler is part of this solution. IIRC spdr enabled Radio/Touch to request mysqueezebox.com to issues a GET to to the player to play a specific URL.

Flash has fallen out of favour and so has Flash streaming (rtmp:// rtmpe://). Support of BBC live and on-demand no longer uses any Flash. I think the BBC Radio Applet in Radio/Touch is now unusable and I think may only support live MP3 streams.

As maintainer of the BBCIPlayer plugin, there have been a few users who have used the Flash protocol support of the plugin to play non BBC RTMP streams. There are only a few legacy RTMP streams now and they can still be handled by LMS and BBCiPlayer plugin. I don't know of anybody using the "native" RTMP of the Radio/Touch but there may be some users who have managed to do so for personal use.

**mherger** · 2020-11-15, 19:12

Slimproto change proposal

> Around the same time Touch and Radio were released and Triode was
> heavily involved in the Squeezeplay development. To support playing BBC
> Radio Live and On-demand BBC using mysqeezebox.com (i.e. no local LMS),
> Triode wrote a BBC Radio Applet and also re-implemented RTMP protocol
> support (in C and not LUA I think) as part of Touch/Radio firmware. The
> "spdr://" protocol handler is part of this solution. IIRC spdr enabled
> Radio/Touch to request mysqueezebox.com to issues a GET to to the player
> to play a specific URL.

My limited research (on a tablet only...) this morning had pretty much
shown me what bpa wrote. RTMP was there to be used with mysb.com _only_,
and for SP based players only, too. I'd say it's a dead protocol, and
that bit in the protocol certainly could be re-used. I'd actually even
say remove references to it on LMS code, update the documentation to
represent the HTTPS use case.

(I was even wondering whether the spdr related code should be removed,
as I removed all mysb.com only related code in 7.8 already. At least
what I found back then).

I think you should be able to implement slimprotoFlags() in the https
protocol handler to set the flag.

I plan to release LMS 8.0.0 soon. This change would be a candidate for
8.1+. I believe that should be ok, as the client side will need some
work, too.

--

Michael

**philippe_44** · 2020-11-15, 19:37

So you both seem to approve the ‘shoehorn’ solution (which does not mean ‘bad’), let’s go for that

Maybe 2 questions

- knowing this is a core flag, I have mixed feeling about letting slimprotoflags method set them. On one hand I agree I can be logical, but I was rather qualifying that as a plugin/extension method.

- the ‘shoehorn’ solution is ultra simple. Shouldn’t we try to put in 8.0 because that will gain a lot of time for broader adoption, like a year probably. I can get that done today

**bpa** · 2020-11-15, 19:45

If we go for the shoehorn solution now which is to be to be replaced by a different solution such as extend slimproto - we should consider what the transition from "shoehorn" to "extended slimproto" will be like if it involves updating clients. Some users have clients from 3rd parties who have integrated squeezelite but may be slow to upgrade.

**philippe_44** · 2020-11-15, 20:06

Originally posted by bpa

If we go for the shoehorn solution now which is to be to be replaced by a different solution such as extend slimproto - we should consider what the transition from "shoehorn" to "extended slimproto" will be like if it involves updating clients. Some users have clients from 3rd parties who have integrated squeezelite but may be slow to upgrade.

I'm not sure I'm following then. The "shoehorn" will be the definite solution for HTTPS. The "extended slimproto" is something we can decide to use if one day we need it, but that would be for other purposes, no?

**bpa** · 2020-11-15, 20:12

Originally posted by philippe_44

I'm not sure I'm following then. The "shoehorn" will be the definite solution for HTTPS. The "extended slimproto" is something we can decide to use if one day we need it, but that would be for other purposes, no?

I think I misunderstood. I though you meant "shoehorn" for a quick solution now and then later when "https" has settled, more time to consider how best to "extend slimproto" we would re-implement https as part of "extended slimproto" and remove "shoehorn".

As I now understand you, implement https with shoehorn and then ALSO later "extend slimproto" but not for the purpose of https so "shoehorn" and "extend slimproto" would co-exist.

**philippe_44** · 2020-11-15, 20:34

Originally posted by bpa

I think I misunderstood. I though you meant "shoehorn" for a quick solution now and then later when "https" has settled, more time to consider how best to "extend slimproto" we would re-implement https as part of "extended slimproto" and remove "shoehorn".

As I now understand you, implement https with shoehorn and then ALSO later "extend slimproto" but not for the purpose of https so "shoehorn" and "extend slimproto" would co-exist.

Yes, but the "extend slimproto" would not be a 8.0 feature as I don't even know yet what we would use it for

Slimproto change proposal

Slimproto change proposal

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment