Home of the Squeezebox™ & Transporter® network music players.
Results 1 to 9 of 9
  1. #1
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,290

    Slimproto change proposal

    I was thinking about the options to inform a player that the URL to be queried is HTTPS instead of HTTP, per discussion here https://github.com/ralph-irving/squeezelite/issues/108.

    Today, slimproto does not tell the player if it should try HTTPS instead of HTTP for direct streaming, although squeezelite (and friends) now support SSL and can inform LMS with the canHTTPS option in "HELO" message. It was not an issue as long as SSL only happens on port 443 in which case squeezelite would try SSL based on the port number sent in "strm_s message". Unfortunately, more and more servers are using SSL on other ports, so we have made workaround to support that and try secured / non-secured but it's a bit cumbersome and has some security flaws.

    Initially, I thought that there was unused bit in some strm_s fields and we could re-use them without breaking compatibility, but I was too optimistic and it seems that none are "really" available. It's also difficult to extend the strm_s message because the string of headers to be sent to the server immediately follow the strm_s header of 24 bytes, so LMS can't offset that string without being sure that the player is fine with that. Similarly, an updated player does not know if LMS has moved it or not, we have the usual complex backward/forward compatibility issue.

    So I wanted to submit here a proposal for two options (keep in mind that no SB device supports SSL, only squeezelite - and friends - does)

    Code:
    # Squeezebox control for tcp stream
    #
    #u8_t command;// [1]'s' = start, 'p' = pause, 'u' = unpause, 'q' = stop, 'f' = flush
    #u8_t autostart;// [1]'0' = don't auto-start, '1' = auto-start, '2' = direct streaming, '3' direct streaming + autostart
    #u8_t mode;// [1]'m' = mpeg bitstream, 'p' = PCM
    #u8_t pcm_sample_size;// [1]'0' = 8, '1' = 16, '2' = 24, '3' = 32
    #u8_t pcm_sample_rate;// [1]'0' = 11kHz, '1' = 22, '2' = 32, '3' = 44.1, '4' = 48, '5' = 8, '6' = 12, '7' = 16, '8' = 24, '9' = 96
    #u8_t pcm_channels;// [1]'1' = mono, '2' = stereo
    #u8_t pcm_endianness;// [1]'0' = big, '1' = little
    #u8_t threshold;// [1]Kb of input buffer data before we autostart or notify the server of buffer fullness
    #u8_t spdif_enable;// [1]  '0' = auto, '1' = on, '2' = off
    #u8_t transition_period;// [1]seconds over which transition should happen
    #u8_t transition_type;// [1]'0' = none, '1' = crossfade, '2' = fade in, '3' = fade out, '4' fade in & fade out, '5' = crossfade-immediate
    #u8_t flags;// [1]
    #               //      0x80 - loop infinitely
    #               //      0x40 - stream without restarting decoder
    #               //      0x20 - Rtmp (SqueezePlay only)
    #               //      0x10 - SqueezePlay direct protocol handler - pass direct to SqueezePlay
    #               //      0x08 - output only right channel as mono
    #               //      0x04 - output only left channel as mono
    #               //      0x02 - polarity inversion right
    #               //      0x01 - polarity inversion left
    #
    #u8_t output_threshold`;// [1]Amount of output buffer data before playback starts in tenths of second.
    #u8_t slaves;// [1]number of proxy stream connections to serve
    #u32_t replay_gain;// [4]replay gain in 16.16 fixed point, 0 means none
    #u16_t server_port;// [2]server's port
    #u32_t server_ip;// [4]server's IP
    #// ____
    #// [24]
    #
    #[HTTP headers to be sent to the server start here]
    1- Shoehorn

    In strm_s, the flags field has a "RTMP" option which I think is not really used as I don't see many services using RTMP these days, but I'm not really sure. My understanding is that this flag was added and only supported by the Radio and/or the Touch. My reading of squeezelite code is that it does not support RTMP (and does not set "canDecodeRtmp" flag in HELO)

    So, we have a exclusive usage here and we could share that bit. If the player supports HTTPS, then LMS can reliably set that flag to indicate that HTTPS is needed even without knowing precisely the model of the player. When it is needed for RTMP (maybe through a plugin) then it's not for squeezelite.

    Any "old" squeezelite version will work as it ignores that RTMP/SSL bit and it will continue to do (for a while ...) the multiple SSL/non-SSL attempts. Any update version of squeezelite (or player that claims to support HTTPS) will look at that bit and open an SSL connection immediately. With time, we could deprecate the squeezelite "funky" code or we could detect LMS version and not use that code. It assumes that no future player will do RTMP *and* SSL (which seems a reasonable assumption)

    This is quick, simple, very located in LMS code but does not provide much future proofness (?)

    NB: there might be an alternative option which is to use the "slaves" byte which seems to be used by rhapsody only and I don't now exactly for what. So we would have exactly the same exclusive situation wrt to player supporting both rhapsody and SSL

    2- Extend slimproto

    In that case, we want to make room in strm_s message's header for future extension, so we need to move (offset) the HTTP headers string. We cannot just put something after the header because squeezelite (and maybe other players) use the hardcoded length of the strm_s header to calculate the length of HTTP headers string.

    So first we would have to let LMS know that players support slimproto extension, which could be done with a "slimproto=XXX" field in "HELO" message. Then, when that flag is set, LMS could offset the header string. I suggest we make that offset a variable length rather than hardcode it. Bear in mind as well that a new squeezelite version, although it has sent "slimproto=2" in it's HELO capabilities, does not know what the LMS server will send in strm_s.

    So LMS, when it has detected a player that supports "extended" slimproto, would add at the end of the 24 bytes of strm_s

    Code:
    #u8_t marker = 0;// [1]must be NULL so that a squeezelite client knows this is not headers string
    #u8_t flags;// [1]to be defined, but at least it contains SSL flag
    #u16_t offset; // [1]      number of bytes to add to find headers string, no including the 2 previous bytes
    #u8_t bytes // [offset]         [... some optional bytes ...]
    #[ HTTP headers ]
    The benefit of that variable offset is to make room for further LMS/strm_s changes without breaking compatibility with existing squeezelite/players version. The "marker" byte allows any squeezelite version to know that this is not the start of headers string. We could consider other fields as well like the version of slimproto/strm_s with which LMS is answering.

    Another benefit of "offset" is that we could add a "can('extendSTRMs')" method for player-type plugins that want to add their own private bits to a strm_s message and we would add retrieve these by calling the extendSTRMs method

    This second option requires much more changes to LMS (it's not horrific, but less straightforward)

    So ... let me know if we should move that discussion to github or not and please bear in mind that I made that up as I was writing it, so please be benevolent for any embarrassing logic mistake I might have made
    Last edited by philippe_44; 2020-11-14 at 21:41.
    LMS 7.9 on Pi 3B+ & Odroid-C2 - SqueezeAMP!, 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, Riva 1 & 3

  2. #2
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    19,619
    I can provide some background to the RTMP implementation and hopefully help in assessing whether it needs to be retained.

    The RTMP implementation is for Flash audio streaming support and not Apple RTMP. It was implemented by Triode who implemented AlienBBC and BBCiPlayer to play BBC radio programs.

    Triode was UK based and around 2005-2010 there was a interest in playing BBC streams as BBC used to use RealAudio but was moving services (TV & Radio) to Flash streaming.

    BBCiPlayer plugin was written to replace AlienBBC and it implemented RTMP in Perl to decode a Flash Audio stream and produce ADTS AAC for the SB players. However this meant playing BBC Radio On-demand and quality live streams required an LMS server.
    Around the same time Touch and Radio were released and Triode was heavily involved in the Squeezeplay development. To support playing BBC Radio Live and On-demand BBC using mysqeezebox.com (i.e. no local LMS), Triode wrote a BBC Radio Applet and also re-implemented RTMP protocol support (in C and not LUA I think) as part of Touch/Radio firmware. The "spdr://" protocol handler is part of this solution. IIRC spdr enabled Radio/Touch to request mysqueezebox.com to issues a GET to to the player to play a specific URL.

    Flash has fallen out of favour and so has Flash streaming (rtmp:// rtmpe://). Support of BBC live and on-demand no longer uses any Flash. I think the BBC Radio Applet in Radio/Touch is now unusable and I think may only support live MP3 streams.

    As maintainer of the BBCIPlayer plugin, there have been a few users who have used the Flash protocol support of the plugin to play non BBC RTMP streams. There are only a few legacy RTMP streams now and they can still be handled by LMS and BBCiPlayer plugin. I don't know of anybody using the "native" RTMP of the Radio/Touch but there may be some users who have managed to do so for personal use.

  3. #3
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,480

    Slimproto change proposal

    > Around the same time Touch and Radio were released and Triode was
    > heavily involved in the Squeezeplay development. To support playing BBC
    > Radio Live and On-demand BBC using mysqeezebox.com (i.e. no local LMS),
    > Triode wrote a BBC Radio Applet and also re-implemented RTMP protocol
    > support (in C and not LUA I think) as part of Touch/Radio firmware. The
    > "spdr://" protocol handler is part of this solution. IIRC spdr enabled
    > Radio/Touch to request mysqueezebox.com to issues a GET to to the player
    > to play a specific URL.


    My limited research (on a tablet only...) this morning had pretty much
    shown me what bpa wrote. RTMP was there to be used with mysb.com _only_,
    and for SP based players only, too. I'd say it's a dead protocol, and
    that bit in the protocol certainly could be re-used. I'd actually even
    say remove references to it on LMS code, update the documentation to
    represent the HTTPS use case.

    (I was even wondering whether the spdr related code should be removed,
    as I removed all mysb.com only related code in 7.8 already. At least
    what I found back then).

    I think you should be able to implement slimprotoFlags() in the https
    protocol handler to set the flag.

    I plan to release LMS 8.0.0 soon. This change would be a candidate for
    8.1+. I believe that should be ok, as the client side will need some
    work, too.

    --

    Michael

  4. #4
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,290
    So you both seem to approve the shoehorn solution (which does not mean bad), lets go for that

    Maybe 2 questions

    - knowing this is a core flag, I have mixed feeling about letting slimprotoflags method set them. On one hand I agree I can be logical, but I was rather qualifying that as a plugin/extension method.

    - the shoehorn solution is ultra simple. Shouldnt we try to put in 8.0 because that will gain a lot of time for broader adoption, like a year probably. I can get that done today
    LMS 7.9 on Pi 3B+ & Odroid-C2 - SqueezeAMP!, 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, Riva 1 & 3

  5. #5
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    19,619
    If we go for the shoehorn solution now which is to be to be replaced by a different solution such as extend slimproto - we should consider what the transition from "shoehorn" to "extended slimproto" will be like if it involves updating clients. Some users have clients from 3rd parties who have integrated squeezelite but may be slow to upgrade.

  6. #6
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,290
    Quote Originally Posted by bpa View Post
    If we go for the shoehorn solution now which is to be to be replaced by a different solution such as extend slimproto - we should consider what the transition from "shoehorn" to "extended slimproto" will be like if it involves updating clients. Some users have clients from 3rd parties who have integrated squeezelite but may be slow to upgrade.
    I'm not sure I'm following then. The "shoehorn" will be the definite solution for HTTPS. The "extended slimproto" is something we can decide to use if one day we need it, but that would be for other purposes, no?
    LMS 7.9 on Pi 3B+ & Odroid-C2 - SqueezeAMP!, 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, Riva 1 & 3

  7. #7
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    19,619
    Quote Originally Posted by philippe_44 View Post
    I'm not sure I'm following then. The "shoehorn" will be the definite solution for HTTPS. The "extended slimproto" is something we can decide to use if one day we need it, but that would be for other purposes, no?
    I think I misunderstood. I though you meant "shoehorn" for a quick solution now and then later when "https" has settled, more time to consider how best to "extend slimproto" we would re-implement https as part of "extended slimproto" and remove "shoehorn".

    As I now understand you, implement https with shoehorn and then ALSO later "extend slimproto" but not for the purpose of https so "shoehorn" and "extend slimproto" would co-exist.

  8. #8
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    6,290
    Quote Originally Posted by bpa View Post
    I think I misunderstood. I though you meant "shoehorn" for a quick solution now and then later when "https" has settled, more time to consider how best to "extend slimproto" we would re-implement https as part of "extended slimproto" and remove "shoehorn".

    As I now understand you, implement https with shoehorn and then ALSO later "extend slimproto" but not for the purpose of https so "shoehorn" and "extend slimproto" would co-exist.
    Yes, but the "extend slimproto" would not be a 8.0 feature as I don't even know yet what we would use it for
    LMS 7.9 on Pi 3B+ & Odroid-C2 - SqueezeAMP!, 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB3. Sonos PLAY:3, PLAY:5, Marantz NR1603, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, Squeezelite on Pi, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5, Riva 1 & 3

  9. #9
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,480

    Slimproto change proposal

    > - knowing this is a core flag, I have mixed feeling about letting
    > slimprotoflags method set them. On one hand I agree I can be logical,
    > but I was rather qualifying that as a plugin/extension method.


    Like all the other flags, yeah. Please feel free to not implement this
    in slimprotoflags.

    > - the ‘shoehorn’ solution is ultra simple. Shouldn’t we try to put in
    > 8.0 because that will gain a lot of time for broader adoption, like a
    > year probably. I can get that done today


    Let's see what we end up with. We can give it a few days try. It would
    be easy enough to pull it should it turn out to be a little more complex.

    --

    Michael

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •