Buildbot proposal: add custom build properties for order and priority

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Buildbot proposal: add custom build properties for order and priority

Ryan Schmidt-24
We currently build ports in a somewhat random order (see https://trac.macports.org/ticket/52766). By default, the next build that buildbot starts is the one that was scheduled first. But if multiple builds happened to get scheduled at the same exact time, the order in which they will build is undefined.

We can fix this by adding a custom "order" property to each build, and modifying our nextBuild function (mentioned in my previous email) so that it looks not only at the scheduled timestamp but also the order property to decide which build to pick.

I don't know whether we currently compute the *correct* order (dependencies should build first) but if not that can be tackled later. A predictable incorrect order is better than a random incorrect order.


I'd also like to be able to assign priorities to builds. Low, Medium, High would be enough. We could add a custom "priority" property to each build for this.

Use case for High priority: suppose hundreds of builds are pending, and you see that a build of a library that lots of ports use is broken, let's say libnetpbm. You want to be able to fix libnetpbm and force it to build right away, before the pending builds go through, because you know many of those pending builds would fail without libnetpbm. Fixing libnetpbm might not even require a commit: libnetpbm might have been marked as failed because of a fetch failure--due to the recent SourceForge outages, for example--and the server issue that caused the fetch failure might have been resolved in the mean time.

Use case for Low priority: suppose we set up a new builder for a new macOS version. We want to eventually build all ports, but it's more important that ports that have just had updates committed are built first. So we would force a build of all ports with Low priority, and while the builder is otherwise idle, it works its way through building all ports, but when a commit comes in, it switches over to building that.

We may not even need to allow the user to specify a priority; we could possibly infer it. For example, if a large number of ports (more than 100, say) are scheduled by a single commit or force, we might infer that it is of low priority. It might be a mass update commit of maintainer lines or checksum types or removing $Id$ lines or adjusting whitespace, and we don't want those builds to block actual port updates. And if a build of a few ports is forced, we might infer that it is of high priority.


Reply | Threaded
Open this post in threaded view
|

Re: Buildbot proposal: add custom build properties for order and priority

Ryan Schmidt-24

On Mar 11, 2018, at 06:01, Ryan Schmidt wrote:

> We currently build ports in a somewhat random order (see https://trac.macports.org/ticket/52766). By default, the next build that buildbot starts is the one that was scheduled first. But if multiple builds happened to get scheduled at the same exact time, the order in which they will build is undefined.
>
> We can fix this by adding a custom "order" property to each build, and modifying our nextBuild function (mentioned in my previous email) so that it looks not only at the scheduled timestamp but also the order property to decide which build to pick.

Thank you very much, Mojca, for fixing this!

The property got called "priority" and right now the nextBuild function only looks at that, not the timestamp. This is fine, so long as portwatcher waits for all portbuilders to finish, as it currently does.


> I don't know whether we currently compute the *correct* order (dependencies should build first) but if not that can be tackled later. A predictable incorrect order is better than a random incorrect order.

Thank you very much, Josh, for fixing this!


> I'd also like to be able to assign priorities to builds. Low, Medium, High would be enough. We could add a custom "priority" property to each build for this.

This is still open.

Reply | Threaded
Open this post in threaded view
|

Re: Buildbot proposal: add custom build properties for order and priority

Mojca Miklavec-2
On 15 March 2018 at 05:31, Ryan Schmidt wrote:
> On Mar 11, 2018, at 06:01, Ryan Schmidt wrote:
>
>> We currently build ports in a somewhat random order (see https://trac.macports.org/ticket/52766). By default, the next build that buildbot starts is the one that was scheduled first. But if multiple builds happened to get scheduled at the same exact time, the order in which they will build is undefined.
>>
>> We can fix this by adding a custom "order" property to each build, and modifying our nextBuild function (mentioned in my previous email) so that it looks not only at the scheduled timestamp but also the order property to decide which build to pick.
>
> Thank you very much, Mojca, for fixing this!
>
> The property got called "priority" and right now the nextBuild function only looks at that, not the timestamp. This is fine, so long as portwatcher waits for all portbuilders to finish, as it currently does.

I semi-forgot about that. We should probably sort by timestamp first.
The question now is how to read the timestamp :)

>> I don't know whether we currently compute the *correct* order (dependencies should build first) but if not that can be tackled later. A predictable incorrect order is better than a random incorrect order.
>
> Thank you very much, Josh, for fixing this!

Thank you very much from me as well.

The variant support from Clemens now needs slight adaptation after
this change. I wanted to test on a local buildbot setup, but I'm not
sure how the newly written function would behave.

https://github.com/macports/mpbb/pull/5

Moca
Reply | Threaded
Open this post in threaded view
|

Re: Buildbot proposal: add custom build properties for order and priority

Ryan Schmidt-24

On Mar 16, 2018, at 05:26, Mojca Miklavec wrote:

>> The property got called "priority" and right now the nextBuild function only looks at that, not the timestamp. This is fine, so long as portwatcher waits for all portbuilders to finish, as it currently does.
>
> I semi-forgot about that. We should probably sort by timestamp first.
> The question now is how to read the timestamp :)

Instead, we might consider transforming "priority" from an integer into a "version number". I bet Python has a built-in version comparison we could use instead of our current numeric comparison operator.

So, instead of priority being "1", "2", "3", it would be "51462.1", "51462.2", "51462.3", where "51462" is the ID number of the portwatcher build that triggered these portbuilders. This has the advantage that we could in future add more components into the priority "version number" (such as a number for low, medium, high priority) without needing to further modify the code that determines which priority comes next.