Herald rule to trigger when Harbormaster build aborted

Question
#1

We use Herald and Harbourmaster to trigger CI builds when a user uploads/updates a diff. At the moment this is a ‘fire and forget’ system, but as we grow we are finding we waste a lot of time building things no one cares about.

We think we can solve half the problem by getting the build to check if the build target is still active and if not cancel the build. This would stop new builds, but wouldn’t cancel running ones.

We already send the build URL to Phabricator as an artefact so were wondering if there was a way (maybe a Herald rule) which could:

  • Trigger when a Build Target hits a given state (e.g. aborted)
  • Have access to build steps and their artefacts
  • Using the URL we sent to Phabricator send an HTTP Request which could cancel the build.

If not is there a way people are currently using to limit obsolete CI builds?

1 Like
#2

You can use the “Abort Older Builds” build step to automatically abort older, incomplete builds of the same object.

For example, if Alice creates D1234, notices a few typos, then updates it several times over the next few minutes, this will normally generate several different builds (for Diff 1, Diff 2, Diff 3, etc, on D1234).

If the build plan for these builds has an “Abort Older Builds” builds step, any existing builds will be aborted first.

Note that there are some caveats here: in most cases, this will only prevent builds which have not yet started from starting. It will not normally abort already-running builds unless you’re running those builds in Phabricator via Drydock. A major reason for this is that most third-party build systems we have bindings for have no API call to abort builds, so there’s nothing the binding could do anyway.

So this will help reduce build pressure if you’re limited by the daemon queue, but probably will not help if you’re using an external, HTTP-based build system and not encountering queue limits.

If you have a build system which you invoke with a “Make HTTP Request” build step, you could possibly use Webhooks to send the system an “Abort” request when the associated build target is aborted today.

In the future, “Make HTTP Request” might gain support for an on-abort request, but it’s generally been simpler to add build system bindings to the upstream rather than make “Make HTTP Request” fancier for all customer use cases so far.

#3

A major reason for this is that most third-party build systems we have bindings for have no API call to abort builds, so there’s nothing the binding could do anyway.

Which build systems are you referring to? Every one I looked at seems to have a way:
Jenkins: How to stop a build in Jenkins via the REST api ? - Stack Overflow
TeamCity: Start and Cancel Builds—TeamCity
GitLab: Jobs API | GitLab
Circle CI: How to programmatically end or cancel the workflow/build? - #2 by fernfernfern - Build Environment - CircleCI Discuss
Travis: Travis CI - API V2 Reference

Cancel Job
POST /jobs/{job.id}/cancel
This request always needs to be authenticated.

I understand that getting from a response to a build ID is difficult in herald as there is no scripting functionality but, at least in Jenkins, we can setup a job which we pass the PHID of the buildable, which then cancels all builds with that buildable as the PHID parameter (this doesn’t even need the URI above).
All we would need to this is:

  • Harbormaster steps to have an ‘on abort’ condition or
  • Herald rule which runs when a Harbormaster rule is aborted.

It would save us (and I think a lot of other people with remote build systems) a lot of time and money building things they don’t care about.
On a personal note, it’s also difficult (and tedious) to explain to our developers why cancelling the Phabricator build doesn’t cancel the CI build - but, x can do that.

It will not normally abort already-running builds unless you’re running those builds in Phabricator via Drydock.
Interesting that Drydock has this ability, but I find it difficult to find any meaningful documentation on Drydoc (e.g. just do this send an HTTP request). It doesn’t help that the docs are littered with keep out warnings:
WARNING: Drydock is very new and has many sharp edges. Prepare yourself for a challenging adventure in unmapped territory, not a streamlined experience where things work properly or make sense.
IMPORTANT: Drydock is not a mature application and may be difficult to configure and use for now.

#4

I’m happy to build this feature if customer interest in it arises. Currently, no customers have expressed interest in it.