Home of the Squeezebox™ & Transporter® network music players.
Page 1 of 6 123 ... LastLast
Results 1 to 10 of 57
  1. #1
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,505

    Plugin Repository Maintenance

    This is a first break-out discussion to the Github organization topic
    (http://forums.slimdevices.com/showthread.php?t=113006).

    As many of you know as of today updating the plugin repository file
    (http://repos.squeezecommunity.org/extensions.xml) is solely in my
    hands. When I'm out, there's no update. This sucks.

    Therefore my plan is to have that repository file managed on Github,
    rather than on a Logitech owned Subversion server. This will allow us to
    share the duties of updating that file.

    That file is created by collecting information from the various plugin
    author's repository files. There's a script to pull them all in, massage
    the data, and write a new extensions.xml file out. But how could we
    further automate the process? I believe there should still be some
    manual step to release the updated file, as all too often a temporary
    hiccup renders a plugin or two inaccessible for a short time. Sure, we
    could fully automate the updat anyway, which would eventually iron out
    the temporary disappearance of some plugins. But I'm not there yet.

    So... the question remains how to automate this as far as possible, yet
    still have a last gate to be passed manually? My thought was to have a
    cron job or similar run the update script every now and then. And then
    automatically create a pull request. Admins would be subscribed to this
    kind of event, would receive a notification, and could sign-off the
    update with a few clicks.

    But the devil is in the details:

    * we don't want to overwhelm these admins, they should only receive one
    notification per update (they currently only happen once every few days)

    * we don't want to pile up open pull requests in case there were
    multiple changes before somebody approved them

    * in order to create pull requests the script would have to push updates
    to some branch: should this be one branch per update, or a dedicated
    branch which would be force overwritten on every update?

    * can GH be configured to only send out a notification on a PR creation,
    but not on its updates?

    I guess the big question is whether somebody has any experience with
    this kind of automation?

    --

    Michael

  2. #2
    Senior Member
    Join Date
    May 2008
    Location
    The Netherlands
    Posts
    122
    I was wondering what the extra manual step accomplishes instead of fully automating the xml generation, and I would think the main thing is testing if the updated plugin loads into LMS. To ease that process I think we should leverage GH Actions so it automatically checks out latest LMS, installs the updated plugin and then checks if the plugins loads. That way the admins have an easy job of assessing the merge requests.

  3. #3
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,505

    Plugin Repository Maintenance

    > I was wondering what the extra manual step accomplishes instead of fully
    > automating the xml generation, and I would think the main thing is
    > testing if the updated plugin loads into LMS.


    No. As I mentioned it's more about the temporary hiccup where a
    repository file would be gone for a moment. That's the biggest issue
    I've been dealing with in the past years.

    --

    Michael

  4. #4
    Senior Member
    Join Date
    May 2008
    Location
    The Netherlands
    Posts
    122
    Quote Originally Posted by mherger View Post
    >No. As I mentioned it's more about the temporary hiccup where a
    repository file would be gone for a moment. That's the biggest issue
    I've been dealing with in the past years.
    Got it, I misunderstood the hiccup. Would that also be a problem if all the plugins are hosted on GH? I guess that all the plugins that are featured in the official repository also get a place on GH?

  5. #5
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,505

    Plugin Repository Maintenance

    > Got it, I misunderstood the hiccup. Would that also be a problem if all
    > the plugins are hosted on GH? I guess that all the plugins that are
    > featured in the official repository also get a place on GH?


    _If_ they were all hosted on github probably not. But you can't expect
    this to happen.

    --

    Michael

  6. #6
    Senior Member erland's Avatar
    Join Date
    Dec 2005
    Location
    Sweden
    Posts
    11,165
    Would it be possible to somehow script it so it’s published automatically as long as the number of plugins are the same or more than before ? And only notify someone through pull request if number of plugins has decreased ?

    I’m thinking that this should take care of the scenario when someone’s repo server is temporarily down and ensure it doesn’t update automatically in this case.

    How many active plugin developers are there ?
    If the number of developers are small and these can be trusted it might even be an option to have a static repository file in a github repo which these have access to and let them update it manually instead of generating it.
    Erland Isaksson (My homepage)
    Developer of many plugins/applets
    Starting with LMS 8.0 I no longer support my plugins/applets (see here for more information )

  7. #7
    Senior Member
    Join Date
    May 2008
    Location
    The Netherlands
    Posts
    122
    Quote Originally Posted by mherger View Post
    * can GH be configured to only send out a notification on a PR creation,
    but not on its updates?
    According to the GitHub plans (https://docs.github.com/en/free-pro-...thubs-products) it is possible in the "Github Team" plan to have (required) pull request reviewers. It only mentions this feature for private repositories, so I do not know if this feature is available for the "GitHub Free for organizations" plan. Everyone can setup their notifications to just notify on "Pull Request reviews" if you want.

  8. #8
    Senior Member
    Join Date
    Nov 2005
    Location
    Berlin, DE
    Posts
    1,909
    Gating release can be done by tests. If there's any hickup in the generating pipeline, we fail the pipeline and wait for the next run. I don't see a need for a human to be involved in this.

    Most CI services have "cron" functionality. Here's an example that we use to do automatic work in our project:

    https://github.com/prometheus/promet....yml#L165-L177

    Other projects I work on use GitHub actions for similar things, including automatic commits to their cosebase:

    https://github.com/coredns/coredns/b...ws/go.tidy.yml

  9. #9
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,505

    Plugin Repository Maintenance

    > Would it be possible to somehow script it so it’s published
    > automatically as long as the number of plugins are the same or more than
    > before ? And only notify someone through pull request if number of
    > plugins has decreased ?


    Excellent idea! "git diff --numstat" should help here.

    > I’m thinking that this should take care of the scenario when someone’s
    > repo server is temporarily down and ensure it doesn’t update
    > automatically in this case.


    Yep.

    > How many active plugin developers are there ?
    > If the number of developers are small and these can be trusted it might
    > even be an option to have a static repository file in a github repo
    > which these have access to and let them update it manually instead of
    > generating it.


    As I mentioned before one of the main reasons for this whole discussion
    was to be able to delegate some of these tasks, and give the community
    (or some select of its members ;-)) more power.

    --

    Michael

  10. #10
    Babelfish's Best Boy mherger's Avatar
    Join Date
    Apr 2005
    Location
    Switzerland
    Posts
    20,505

    Plugin Repository Maintenance

    > Gating release can be done by tests. If there's any hickup in the
    > generating pipeline, we fail the pipeline and wait for the next run. I
    > don't see a need for a human to be involved in this. >
    > Most CI services have "cron" functionality.


    Github actions don't come for free. There's a limited budget of free
    minutes. I don't have much experience yet, but a polling task run hourly
    can easily consume large parts of 2000 minutes/month...

    --

    Michael

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •