Home of the Squeezebox™ & Transporter® network music players.
Results 1 to 9 of 9
  1. #1
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    4,148

    Timers question / bug

    A timer is stopped by some callback and is self-restarted inside his own callback. Is this an un-authorized used/cause for race condition? I though that Perl, inside LMS was cooperative only so functions cannot be interrupted during their linear execution. Is there an exception for HTTP callback (the timer, in his callback, creates a HTTP async and restart itself in the callback of the HTTP response
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5

  2. #2
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,084

    Timers question / bug

    > A timer is stopped by some callback and is self-restarted inside his own
    > callback. Is this an un-authorized used/cause for race condition? I


    It's actually pretty common practice in LMS to kill all timers from
    within the callback. Sometimes we don't want an action to be run more
    than once, but it can be triggered from several places.

    > though that Perl, inside LMS was cooperative only so functions cannot be
    > interrupted during their linear execution. Is there an exception for
    > HTTP callback (the timer, in his callback, creates a HTTP async and
    > restart itself in the callback of the HTTP response


    This part of your question I don't understand.

    --

    Michael

  3. #3
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    4,148
    Quote Originally Posted by mherger View Post
    > A timer is stopped by some callback and is self-restarted inside his own
    > callback. Is this an un-authorized used/cause for race condition? I


    It's actually pretty common practice in LMS to kill all timers from
    within the callback. Sometimes we don't want an action to be run more
    than once, but it can be triggered from several places.


    I’m confused (I don’t have code under my eyes - I might have had a brain fart when I wrote that code) because I thought timers, when set, only elapse once, they are not recurring. So the question was is it fine to restart the timer from within the timer callback (to make it recurring) not to kill it from there. Killing the timer in its callback would not be needed, because if you’re there it’s already elapsed. Now another part of code actually does the killing when we don’t want the timer to be recurring anymore
    > though that Perl, inside LMS was cooperative only so functions cannot be
    > interrupted during their linear execution. Is there an exception for
    > HTTP callback (the timer, in his callback, creates a HTTP async and
    > restart itself in the callback of the HTTP response
    So per my comment above, the timer is killed not inside its own callback, but by another function. Instead, it is a recurring timer which is restarted by its own callback. What I was wondering is that in normal threaded code, you have to protect these 2 pieces of code because the timer might interrupt any part of your code. So if you access some context-shared variable between the timer callback and the ‘killing’ function, you have to use semaphore. I though that in LMS was all cooperative so, no préemption can happen, hence no protection required: the timer callback cannot interrupt or be interrupted by any other piece of code, accessing shared context is save
    Now, the more specific part of my question was that the part killing the timer is in an HTTP async callback, so I wanted to double that even there, preemption is not possible

    Hope this clarifies



    This part of your question I don't understand.

    --

    Michael[/QUOTE]
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5

  4. #4
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,084

    Timers question / bug

    > fart when I wrote that code) because I thought timers, when set, only
    > elapse once, they are not recurring. So the question was is it fine to


    That's correct.

    > restart the timer from within the timer callback (to make it recurring)


    That's another popular pattern and totally ok.

    > not to kill it from there. Killing the timer in its callback would not
    > be needed, because if you’re there it’s already elapsed. Now another
    > part of code actually does the killing when we don’t want the timer to
    > be recurring anymore


    My wording probably wasn't accurate: a timer only lives once, of course.
    But several timers can point to the same callback. That's the use case I
    was referring to.

    > which is restarted by its own callback. What I was wondering is that in
    > normal threaded code, you have to protect these 2 pieces of code because
    > the timer might interrupt any part of your code. So if you access some
    > context-shared variable between the timer callback and the ‘killing’
    > function, you have to use semaphore. I though that in LMS was all
    > cooperative so, no préemption can happen, hence no protection required:
    > the timer callback cannot interrupt or be interrupted by any other piece
    > of code, accessing shared context is save


    That's mostly correct: LMS is cooperative multitasking. That said: any
    block of code can yield control control to other code by calling
    main::idle() or main::idleStream(). This is used across the code to
    prevent stalling the stream when longish blocks of code are run.

    > Now, the more specific part of my question was that the part killing the
    > timer is in an HTTP async callback, so I wanted to double that even
    > there, preemption is not possible


    It's still not clear what you're talking about. Would you have some code
    to show? Or a link in to our code on github where this happens?

    --

    Michael

  5. #5
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    16,265
    Quote Originally Posted by philippe_44 View Post
    I though that in LMS was all cooperative so, no préemption can happen, hence no protection required: the timer callback cannot interrupt or be interrupted by any other piece of code, accessing shared context is save
    Now, the more specific part of my question was that the part killing the timer is in an HTTP async callback, so I wanted to double that even there, preemption is not possible
    AFAIK and have not experienced anythgin to contrary even when using non LMS http 1.1 via AnyEvent. LMS uses perl evloop so it is single threaded. The standard hhtp code uses select to check for event that needs process such as data waiting, etc. If timer has expired or http has completed - timer does not preempt as it run from same evloop.

    There is one exception - some parts of the code uses Proc::Background which are urn in a separate process and these are usually self contained tasks.
    Last edited by bpa; 2018-10-11 at 07:45.

  6. #6
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    4,148
    Quote Originally Posted by mherger View Post

    It's still not clear what you're talking about. Would you have some code
    to show? Or a link in to our code on github where this happens?

    --

    Michael

    Quote Originally Posted by bpa View Post
    AFAIK and have not experienced anythgin to contrary even when using non LMS http 1.1 via AnyEvent. LMS uses perl evloop so it is single threaded. The standard hhtp code uses select to check for event that needs process such as data waiting, etc. If timer has expired or http has completed - timer does not preempt as it run from same evloop.

    There is one exception - some parts of the code uses Proc::Background which are urn in a separate process and these are usually self contained tasks.
    Thanks to both of you.

    Michael, this is my update of the YT plugin to support dash and mp4. The problem I'm having seems to be around unexpected interaction, in ProtocolHandler.pm, between the "sysread" and "updateMPD". I realize that, even in cooperative mode, the line 233 and 243 work on the same context that can have changed between 233 and 243, but that should not create an issue.

    BTW, @bpa thanks again for the mp4 code. Something I noticed: some of the files on YT have a large quantity of very small 'moof' atoms and in that case, the 'trun' atom does no contain 'sample_size'. The 'default_sample_size' found in the atom 'tfhd' must be used, which (I think) was not the case in your code, as the function that build ADTS frames is only using "trun' atom
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5

  7. #7
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    16,265
    Quote Originally Posted by philippe_44 View Post
    BTW, @bpa thanks again for the mp4 code. Something I noticed: some of the files on YT have a large quantity of very small 'moof' atoms and in that case, the 'trun' atom does no contain 'sample_size'. The 'default_sample_size' found in the atom 'tfhd' must be used, which (I think) was not the case in your code, as the function that build ADTS frames is only using "trun' atom
    I'm glad the code is of use. The MP4 code was written as I got to understand ISOBMFF MPEG4 spec and also looked at ffmpeg MPEG4 code as a sort of reference. I can't remember the details but getting to use trun to make ADTS frames was key for LMS. I know my implementation of DASH and streaming MPEG4 is not complete and may have made assumptions (e.g sample size is always in trun) necessary to make it work. The implementation was only intended to be sufficient for BBC's DASH/MPEG4 variant which since has had a few special tweak for flac and spatial audio. Discovering the nuances of the spec is one the "special" joys - in the end do what you have to do to make it work.

  8. #8
    Senior Member
    Join Date
    May 2008
    Location
    Canada
    Posts
    4,148
    Quote Originally Posted by bpa View Post
    I'm glad the code is of use. The MP4 code was written as I got to understand ISOBMFF MPEG4 spec and also looked at ffmpeg MPEG4 code as a sort of reference. I can't remember the details but getting to use trun to make ADTS frames was key for LMS. I know my implementation of DASH and streaming MPEG4 is not complete and may have made assumptions (e.g sample size is always in trun) necessary to make it work. The implementation was only intended to be sufficient for BBC's DASH/MPEG4 variant which since has had a few special tweak for flac and spatial audio. Discovering the nuances of the spec is one the "special" joys - in the end do what you have to do to make it work.
    Yes, this spec is pretty hairy and DASH in addition to MP4 file format is a real pain. I was starting to go through MP4 specs, trying to understand it and how to extract audio and transform it to ADTS then I got your code which 1/helped me a lot to understand the spec 2/saved me a huge amount of of time in implementation.

    All that reminded me why I wanted to stay away from that after I went to then other pain of webM format, but unfortunately, YT is pushing me there.

    These standards always have that problem (I work on the 3GPP telecom side), where the writers are creative with options and various ways to do the same thing, but we (I'm on the other side in telecom) often forget that it's easy for the server side as you chose what sub-part to implement, but the client has no choice other than implementing all of them. DASH is a good example of that, the way segment, segmentTimeline and segmentTemplate can be described ... BBC went segment template, YT went full segment timeline, I read a lot about the disagrements wrt best way to reach live edge. I hope YT will not go to segment template which would force we to get UTC time and template pattern analysis ...
    LMS 7.7, 7.8 and 7.9 - 5xRadio, 3xBoom, 4xDuet, 1xTouch, 1 SB2. Sonos PLAY:3, PLAY:5, Marantz NR1603, JBL OnBeat, XBoxOne, XBMC, Foobar2000, ShairPortW, JRiver 21, 2xChromecast Audio, Chromecast v1 and v2, , Pi B3, B2, Pi B+, 2xPi A+, Odroid-C1, Odroid-C2, Cubie2, Yamaha WX-010, AppleTV 4, Airport Express, GGMM E5

  9. #9
    Senior Member
    Join Date
    Oct 2005
    Location
    Ireland
    Posts
    16,265
    Quote Originally Posted by philippe_44 View Post
    These standards always have that problem (I work on the 3GPP telecom side), where the writers are creative with options and various ways to do the same thing, but we (I'm on the other side in telecom) often forget that it's easy for the server side as you chose what sub-part to implement, but the client has no choice other than implementing all of them. DASH is a good example of that, the way segment, segmentTimeline and segmentTemplate can be described ... BBC went segment template, YT went full segment timeline, I read a lot about the disagrements wrt best way to reach live edge. I hope YT will not go to segment template which would force we to get UTC time and template pattern analysis ...
    At least with the 3GPP experience you know how to read protocol specs. There are so many possible features in DASH such as audio inserts, syncing and dynamic bandwidth - I think it is sensible to just do what the specific use case (e.g. YT, BBC) requires. I hated the timeline stuff - trying to get a "Live edge" suitable for buffered player was messy (I got "comments" from BBC). I didn't read much on full sgement timeline but UTC and templates weren't bad - just SB users wonder why does the DASH stream start x secs later than FM or even MP3. Part of the "challenge" was to do it all in Perl so that no executables with many builds were required - Triode's Flash/RTMP & HLS implementations showed the way with that.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •