Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 6 123 ... LastLast
Results 1 to 10 of 55
  1. #1
    Senior Member
    Join Date
    Jan 2011
    Location
    Germany
    Posts
    226

    How to best send asynchronous HTTP client requests in C?

    I am trying to adapt philippe44's LMS-to-* Plugins to a hue system.

    These systems are discovered using UPNP's discovery. After that the communication is based on plain HTTP requests (GET, POST, SEND,...).
    The resulting bridge would be a kind of player that gets a sound stream, analyzes it and steers the light - well, that's at least the idea. All is JSON based...

    Currently I am adapting the LMS-to-UPNP bridge of philippe44 as it covers the upnp part for the discovery very well. I am in a state that the bridge is discovered and a player is exposed to LMS.

    What I have to do next is to communicate to the bridge via the http-request and I am actually stuck (--> head aching: asynchrinous networking for LMS, threads, upnp, http,... too much new things).

    A request sent to the hue would be:
    Code:
    POST /api/newdeveloper HTTP/1.1
    {"devicetype":"my_hue_app#iphone peter"}
    The response would be something like:
    Code:
    HTTP/1.1 200 OK
    Server: ...
    Content-Length: ...
    Content-Language: ...
    Connection: close
    Content-Type: application/json
    <JSON HERE>
    The detailed and hopefully error free explanation is displayed very well here: https://developers.meethue.com/docum...etting-started

    I thought of putting the http-communication (request, response) together with the JSON parsing into a dedicated function for reuse in the bridge.

    Can anybody help me with implementing the "threaded-asynchronous http-requesting-responsing-and-parsing" part?
    Are there any libraries which I could use?

    As far as I understand the communication is plain and simple, but the implementation respecting async and thread not.

    A very, very early code repo can be found here: https://github.com/chincheta0815/Hue...er/application

  2. #2
    Senior Member meep's Avatar
    Join Date
    Aug 2007
    Location
    Ireland
    Posts
    187
    Quote Originally Posted by chincheta0815 View Post
    I am trying to adapt philippe44's LMS-to-* Plugins to a hue system.

    These systems are discovered using UPNP's discovery. After that the communication is based on plain HTTP requests (GET, POST, SEND,...).
    The resulting bridge would be a kind of player that gets a sound stream, analyzes it and steers the light - well, that's at least the idea. All is JSON based...

    Currently I am adapting the LMS-to-UPNP bridge of philippe44 as it covers the upnp part for the discovery very well. I am in a state that the bridge is discovered and a player is exposed to LMS.

    What I have to do next is to communicate to the bridge via the http-request and I am actually stuck (--> head aching: asynchrinous networking for LMS, threads, upnp, http,... too much new things).

    A request sent to the hue would be:
    Code:
    POST /api/newdeveloper HTTP/1.1
    {"devicetype":"my_hue_app#iphone peter"}
    The response would be something like:
    Code:
    HTTP/1.1 200 OK
    Server: ...
    Content-Length: ...
    Content-Language: ...
    Connection: close
    Content-Type: application/json
    <JSON HERE>
    The detailed and hopefully error free explanation is displayed very well here: https://developers.meethue.com/docum...etting-started

    I thought of putting the http-communication (request, response) together with the JSON parsing into a dedicated function for reuse in the bridge.

    Can anybody help me with implementing the "threaded-asynchronous http-requesting-responsing-and-parsing" part?
    Are there any libraries which I could use?

    As far as I understand the communication is plain and simple, but the implementation respecting async and thread not.

    A very, very early code repo can be found here: https://github.com/chincheta0815/Hue...er/application
    is it absolutely necessary to do the communications in C?

    ive just finished building something similar for integration with an Alexa skill where my plugin makes async calls to both the cloud-based skill and the LMS itself. It's all done inside the plugin using perl and libs already available.

    Happy to share the code and give you a summary (though there's a lot of other stuff going on in the plugin so isolating just the async http might not be the easiest task).


    ALEXA LMS SKILL: http://www.hab-tunes.com | Twitter: #habtunes
    Personal HA BLOG: http://mediaserver8.blogspot.com

    Squeezebox | Squeezebox Radio x 2 | Squeezebox Duet

  3. #3
    Senior Member
    Join Date
    Jan 2011
    Location
    Germany
    Posts
    226
    Quote Originally Posted by meep View Post
    is it absolutely necessary to do the communications in C?
    I think yes, because the plugin/bridge needs to do:
    1. UPNP discovery and connection (could be done in perl, but LMS-to-UPNP has that implemented yet).
    2. Steer the hue system via http requests & responses using JSON (also possible in perl)
    3. Get a sound stream from LMS and analyze it concerning e.g. VU-metrics for adjusting the light (LMS-to-* get sound via squeezetiny)
    4. Maybe get everything in group sync, do recoding, etc...
    4. Be fast in doing all that (I think of threading etc.)

    The idea: Create a 'fake' player being able to play lights not sound, meaning turn the 'streaming' part into an 'analyze and adjust' part. Say: Media Renderer -> Light Renderer.
    The charming thing: Philippe44's bridges can do all that, but there is none doing it for hue ;o)

    The non-charming thing: I am very good in adjusting, compiling on different systems, debugging & understanding. But not in designing new stuff as I am no deeply experienced software engineer. There is simply too much new stuff like threads, async communication, etc. But I thing with advice, code fragments, examples & explanation I should be able to get that working. Once I got that working I would then cleanup the code etc...

    For now I got the binary discover the hue bridge and make it compile on solaris. Now I have to do the communication routines with the hue using http.
    Under perl I already got the hue connected and was able to steer the bulbs...

  4. #4
    Senior Member
    Join Date
    Jan 2011
    Location
    Germany
    Posts
    226
    Maybe some addition about my issue:
    The main process of opening a socket, writing to it and waiting for a response is quite clear to me. The confusing is to implement that process properly into a async, non-blocking & threaded program.

  5. #5
    Senior Member pippin's Avatar
    Join Date
    Oct 2007
    Location
    Berlin
    Posts
    13,993
    If you really need to use C, I'd bite the bullet and use libCurl. Does everything you'd ever need WRT http communication.
    ---
    learn more about iPeng, the iPhone and iPad remote for the Squeezebox and
    Logitech UE Smart Radio as well as iPeng Party, the free Party-App,
    at penguinlovesmusic.com
    New: iPeng 9, the Universal App for iPhone, iPad and Apple Watch

  6. #6
    Senior Member
    Join Date
    Jan 2011
    Location
    Germany
    Posts
    226
    Quote Originally Posted by pippin View Post
    If you really need to use C, I'd bite the bullet and use libCurl. Does everything you'd ever need WRT http communication.
    That was also in my thoughts, but I hope to get it more lightweight, since I think the communication is not too complex. Another thing is that avoiding libCurl would make the binary fairly easy to maintain.

    Actually there are lots of examples of opening sockets, sending and waiting for responses, but the threading and the non-blocking makes me thinking...

  7. #7
    Senior Member pippin's Avatar
    Join Date
    Oct 2007
    Location
    Berlin
    Posts
    13,993

    How to best send asynchronous HTTP client requests in C?

    Well, it's your code.
    I'd not do it.
    HTTP comms is not that simple, then you might add other feature down the road and in the end you end of maintaining just another breed of limited-functionality http stack.
    libCurl might be a bit heavy but it's available on pretty much every platform, easy and straightforward to use, well-maintained and will support whatever other functionality you might need down the road, like https, tunneling, caching or whatever.
    ---
    learn more about iPeng, the iPhone and iPad remote for the Squeezebox and
    Logitech UE Smart Radio as well as iPeng Party, the free Party-App,
    at penguinlovesmusic.com
    New: iPeng 9, the Universal App for iPhone, iPad and Apple Watch

  8. #8
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    3,008
    Quote Originally Posted by chincheta0815 View Post
    I am trying to adapt philippe44's LMS-to-* Plugins to a hue system.

    These systems are discovered using UPNP's discovery. After that the communication is based on plain HTTP requests (GET, POST, SEND,...).
    The resulting bridge would be a kind of player that gets a sound stream, analyzes it and steers the light - well, that's at least the idea. All is JSON based...

    Currently I am adapting the LMS-to-UPNP bridge of philippe44 as it covers the upnp part for the discovery very well. I am in a state that the bridge is discovered and a player is exposed to LMS.

    What I have to do next is to communicate to the bridge via the http-request and I am actually stuck (--> head aching: asynchrinous networking for LMS, threads, upnp, http,... too much new things).

    A request sent to the hue would be:
    Code:
    POST /api/newdeveloper HTTP/1.1
    {"devicetype":"my_hue_app#iphone peter"}
    The response would be something like:
    Code:
    HTTP/1.1 200 OK
    Server: ...
    Content-Length: ...
    Content-Language: ...
    Connection: close
    Content-Type: application/json
    <JSON HERE>
    The detailed and hopefully error free explanation is displayed very well here: https://developers.meethue.com/docum...etting-started

    I thought of putting the http-communication (request, response) together with the JSON parsing into a dedicated function for reuse in the bridge.

    Can anybody help me with implementing the "threaded-asynchronous http-requesting-responsing-and-parsing" part?
    Are there any libraries which I could use?

    As far as I understand the communication is plain and simple, but the implementation respecting async and thread not.

    A very, very early code repo can be found here: https://github.com/chincheta0815/Hue...er/application
    Hi again :-)

    For JSON: if the request/reponse is simple, you can build/parse the payload manually. If not, you can use jansson (https://github.com/akheron/jansson) which works very well for me. I use it in my chromecast plugin.

    For HTTP: as discussed, if this is simple enough, you can simply build the request as a string (potentially with jansson to build the payload) and "send" it. Handle response, through select/recv, you can parse using https://github.com/nodejs/http-parser, although it might enough to just grab the body. The Shairtunes2W plugin uses it and shows an example how to use it. Personally, I would re-use the MRThread that exists for each UPnP player to handle HTTP exchanges with the Hue "player" associated with the LMS player. The socket for each HTTP exchange would be opened/closed here (don't open a socket in a thread and close it in another, I think it's generally a bad idea). When a request has to be created as the result of a LMS query (received in sq_callback), I would queue an action for that MRthread. This is what I do in the MRThread of my castbridge. It guarantees "isolation" between the LMS/Squeezelite part and the Player/Cast (Hue in your case). The MRThread waits on a signal that is emitted upon queuing and then you can send the HTTP request and use select to wait for the answer

    You'll need PCM to be able to do any signal processing with the sound, but I'd leave that for later. At the beginning, switching on/Off the Hue with the play/stop would probably be a good first step. A simple HTTP GET/POST sequence I guess, likely without the need of jansson or http_parser to build things step by step
    Last edited by philippe_44; 2017-04-16 at 14:28.
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express

  9. #9
    Senior Member
    Join Date
    Jan 2011
    Location
    Germany
    Posts
    226
    Quote Originally Posted by philippe_44 View Post
    Personally, I would re-use the MRThread that exists for each UPnP player to handle HTTP exchanges with the Hue "player" associated with the LMS player. The socket for each HTTP exchange would be opened/closed here (don't open a socket in a thread and close it in another, I think it's generally a bad idea).
    So every commands are issued in there and I do not have to care about blocking etc? Cool!
    So I would do a function for that and try the best.

    For design purposes: Each hue has to be checked for registration once. As far as I understand the code that should go into AddMRDevice. Do I also not have to be concerned about the threads putting it there?

    Quote Originally Posted by philippe_44 View Post
    You'll need PCM to be able to do any signal processing with the sound, but I'd leave that for later. At the beginning, switching on/Off the Hue with the play/stop would probably be a good first step. A simple HTTP GET/POST sequence I guess, likely without the need of jansson or http_parser to build things step by step
    No worry. Bit by bit ;o)

  10. #10
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    3,008
    Quote Originally Posted by chincheta0815 View Post
    So every commands are issued in there and I do not have to care about blocking etc? Cool!
    So I would do a function for that and try the best.
    Yes, that's the idea. Otherwise you're blocking the thread in charge of handling the control communication with LMS and that's not good.
    For design purposes: Each hue has to be checked for registration once. As far as I understand the code that should go into AddMRDevice. Do I also not have to be concerned about the threads putting it there?
    No worry. Bit by bit ;o)
    I'm not sure I understand exactly your question, but in AddMRDevice, you should not be concerned by blocking here. There is a master thread (MainThread) that is doing a bit of houskeeping and at program interval, it launches a UPnP search. When the result of that search happens, a temporary thread is started to add/delete players based on the search. This is where AddMRDevice happen, so you have the time you need in this thread, unless you take more than the time between two searches... Just be careful to use mutex every time you need to access device data and there is a potential conflict.

    In general, you unfortunately need to do a bit of mashup of UPnP, Cast and AirPlay bridges because most of the HTTP calls to be sent by UPnP players are handled by libupnp, but that does bot work for chromecast communication, so I added "shims" in Cast and AirPlay bridges. But you probably do not need to do all of that. I can't write a whole design doc, but a few informations still.

    I put aside the threads that come from squeezelite. Here, for *each* player you have
    - one for LMS protocol exchange/control,
    - one for streaming audio data,
    - one for audio decoder (only in airplay)
    - one for either formatting the output and waiting for the player grab it (upnp and cast) or actually sending the data (airplay)

    For the "other side", there is one master thread for some house keeping, scanning for new players and starting a temporary thread (as explained before) to handle search results. Then you have bunch of other threads per player

    In UPnPBridge
    -1) one thread per player that does the polling of the player's state
    -2) a callback (sq_callback) that is called by the squeezelite control side and must *not* be blocking
    - UPnP requests are sent by either by sq_callback or the polling thead, but they are never blocking thanks to libupnp
    - the libupnp has its own way to handle such requests through threads and jobs and then when they receive responses, it uses user callbacks like CallbackActionHandler and CallbackEventHandler which have to handle the results
    - libupnp also provided a webserver which is used by the player to retrieve the actual audio data. That webserver information is populated by the output/formatting squeezelite thread - you do not need any of that

    in CastBridge
    -1) and 2) are still used but 1) wakes-up either on signal or on timer
    - there is no help from the libupnp to handle the communication with the CC device and these CC devices need a permanent SSL connection for control, so another thread per CC is taking care of that: CastSocketThread 3)
    - the audio data is gathered by the CC through a webserver as well and again, you don't need that
    - when 1) or 2) want to send a command to a CC device, it's queued most of the time (unless no response is expected: nothing pending). Then CastSocketThread is in charge of managing that queue of request/response (when a command is queued, a signal is triggered)
    - when CastSocketThread has received a response, it puts it in another queue and triggers a signal for the thread 1) which will then take approppriate action.
    - These queues are what garantees non-blocking

    in AirplayBridge, things are similar but it's a full audio decoder and the audio data is directly pushed to the player, it's not buffered in a webserver, waiting for the player to get it (that's just the way AirPlay works)
    - 1) and 2) are still used, but 1) does more work, it sort of does what 1) and 3) do together
    - In 2) (sq_callback), blocking still shall not happen. So a request from LMS is queued an a signal is generated and 1) is managing that queue and transforming that to proper AirPlay commands, that I won't describe here

    If I were you I'd use and hybrid solution where in 2) I would queue requests to be handled by 1), like they are in AirPlay.
    - in 1) I would have a state where I'm waiting for a signal (or a timeout). When something needs to be processed from the queue, I would build the HTTP request, open a socket, send it and then enter a state where I wait for the response on that socket. Once reponse has been received or after timeout, I would close the socket and return to first state.
    - You might want to have 2 threads, like what I have in CastBridge, but I would recommend doing that later and if needed. To start with, just one thread with 2 states is probably good enough and does not add to complexity.
    Last edited by philippe_44; 2017-04-16 at 15:48.
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •