Podcast private feed not playing

[daveaudi0]

What Host header is being generated by your LMS ? I once came across a HTTP server that oddly refused to support UserAgent with "iTunes" in it - when notified the operator fixed their system.

It's using <domain of the URL>:443 as the Host header, at least that's what I see in the log file. I'm not entirely sure if this is a proper value for the Host header and therefore I don't know whose issue it is - LMS or the webhost of the podcast. And if it is an LMS issue, I'm not sure how to resolve it, other than getting into the source code and modifying how it builds HTTP requests.

[mherger]

You call these "private" feeds. Does that mean you have control over the
server side?

You seem to be using AWS to host the service. It's super easy to cause
these kinds of problems eg. with AWS' WAF (web application firewall)
you'd put in front of your service to protect it from "abuse". It's
probably worth looking into what kind of security measures you have on
the service.

--

Michael

[daveaudi0]

You call these "private" feeds. Does that mean you have control over the
server side?

You seem to be using AWS to host the service. It's super easy to cause
these kinds of problems eg. with AWS' WAF (web application firewall)
you'd put in front of your service to protect it from "abuse". It's
probably worth looking into what kind of security measures you have on
the service.

--

Michael

Private meaning that they are subscription based services that provide individualized RSS feeds to members. I do not have control of the service, I am only given a URL. In this case, it looks like this particular podcast host uses Supercast as the provider. And Supercast uses AWS.

However, at this point I am still unsure of this being their issue, or an LMS issue.

[Paul Webster]

It's using <domain of the URL>:443 as the Host header, at least that's what I see in the log file. I'm not entirely sure if this is a proper value for the Host header and therefore I don't know whose issue it is - LMS or the webhost of the podcast. And if it is an LMS issue, I'm not sure how to resolve it, other than getting into the source code and modifying how it builds HTTP requests.

Does the host name change between the original request and the redirect?

[daveaudi0]

Does the host name change between the original request and the redirect?

Yes, it redirects to a different domain. I don't think this is an issue though; the request url is to the redirected domain as well, so it matches the Host header. Also, when specifying the original url as the Host header, it too fails.

One thing I discovered while troubleshooting with curl is that the request will be successful in 2 situations: 1) no Host header specified, and 2) a Host header set to <domain of the URL>, not <domain of the URL>:443. So having the port in the Host header is problematic. This leads me to believe that this is now an LMS issue, and it should not build requests with port numbers in the Host header.

[mherger]

> One thing I discovered while troubleshooting with curl is that the
> request will be successful in 2 situations: 1) no Host header specified,
> and 2) a Host header set to <domain of the URL>, not <domain of the
> URL>:443. So having the port in the Host header is problematic.

Hmm... I believe it's correct according to spec to have the port in that
header:

https://developer.mozilla.org/en-US/...P/Headers/Host

--

Michael

[daveaudi0]

Ok I am pretty sure I found the issue. Debugging both the Networking and Player components of LMS was the key.

As seen in the attached log, the Networking requests return 200 OK, while the Player requests return 403 Forbidden. The reason is the way Player builds it's requests, which is slightly different than Networking. Player adds a port to the Host header if it is anything other than port 80. But Neworking only adds a port to the Host header if it is not default to the specific protocol (http, https). And this behavior can be seen in the log: no port on the Host header in Networking requests; port 443 added to the Host header on the Player requests. (Networking also adds additional Cookie headers, but that is beyond the current issue).

Now, the reason this returns 403 Forbidden may only be specific to my case. However, I think it would be best practice for LMS to create requests as similar as possible between these two components. Therefore I think Player should be modified to resemble Networking in the way the Host header is built. I may try to create a PR for this issue. It probably would have been more appropriate to post this thread on github, but thank you everyone for your help and suggestions.

[Paul Webster]

I wonder if the redirect from Amazon has predicted what certain header values will be and then used them to generate the X-Amz-SignedHeaders field ...
and then when what comes back for certain header fields does not match so it fails.

I say this because the RFC has a piece that says:

If the target URI includes an authority component, then a
   client MUST send a field-value for Host that is identical to that
   authority component, excluding any userinfo subcomponent and its "@"
   delimiter (Section 2.7.1).  If the authority component is missing or
   undefined for the target URI, then a client MUST send a Host header
   field with an empty field-value.

RFC7230 Section 5.4
https://tools.ietf.org/html/rfc7230#section-5.4

where "authority" is the

[username:password@]hostname[:port]

part

In this special case the remote host knows what the target should be and that it did not put :443 in it so it could, theoretically at least, check that it is not there when it comes back in the HOST header (because it had built a checksum/hash of what it expected in certain fields).

If that is what is happening always removing the port from that header field when it is the default for that protocol will work in this particular case but, if the theory is right, could fail in another if the remote end had specifically used https AND included the port number in the redirect.

I might well be over-thinking this ... but at least once the simple fix is in place, if other users start reporting connections failing that fail until this change is rolled back then this might be the reason.

[Paul Webster]

Looks like this is indeed how it works ... (X-Amz-SignedHeaders being the list of header fields and X-Amz-Signature is the result of their pre-signing calculation.
https://docs.aws.amazon.com/AmazonS3...ring-auth.html

X-Amz-SignedHeaders	
Lists the headers that you used to calculate the signature. The following headers are required in the signature calculations:

The HTTP host header.

Any x-amz-* headers that you plan to add to the request.

Note
For added security, you should sign all the request headers that you plan to include in your request.

[mherger]

> As seen in the attached log, the Networking requests return 200 OK,
> while the Player requests return 403 Forbidden. The reason is the way
> Player builds it's requests, which is slightly different than
> Networking. 'Player adds a port to the Host header if it is anything

What kind of player are you using? I must assume it's not a SB, as that
wouldn't support https.

Would forcing proxied streaming in LMS work?

> Now, the reason this returns 403 Forbidden may only be specific to my
> case. However, I think it would be best practice for LMS to create
> requests as similar as possible between these two components. Therefore
> I think 'Player'
> (https://github.com/Logitech/slimserv...tocols/HTTP.pm)
> should be modified to resemble 'Networking'
> (https://github.com/Logitech/slimserv.../Async/HTTP.pm)
> in the way the Host header is built. I may try to create a PR for this
> issue. It probably would have been more appropriate to post this thread
> on github, but thank you everyone for your help and suggestions.

I personally believe this is a good place, as you'd find more people
here who can help. I'll be happy to merge your PR. Thanks!

--

Michael