Run different build plans for reviews in same project?

#1

We use Mercurial, and it seems that Mercurial support in Arcanist/Phabricator kinda sucks and tries very hard to imitate GitFlow, which is totally not how our repository works. Thus, I might be asking about some rather unorthodox Phabricator usage, so please bear with me and don’t hesitate to suggest alternative solutions (within a reasonable, so no VCS migration).


We have a Jenkins job, say check-build - it simply checks that the project is compiled successfully, produces all the needed binaries and all the tests succeed. It operates on Differential diff ID to arc patch changes from a revision and check they won’t break anything too hard.

We also have couple other Jenkins jobs, say build-windows and build-linux that perform a “full build” with a lot of additional steps, like pulling certificates, signing stuff, preparing installers/RPM packages etc. For this job user can also specify a diff ID, so they would get a “custom” build with their changes that are not yet pushed upstream.

We have a Harbormaster Build plan to run check-build and a Herald rule to trigger it on any incoming revisions (ideally; right now it is tested only on my revisions). I would add to it some limitation like “incoming revisions with X status”, with X being something like not 'Changes Planned' - so people would simply run arc diff --plan-changes, but maybe there would be a better idea.

Is there a way to somehow differ revisions (same project, same development team), so if someone creates one to perform a custom build from their revision, there would be only a single Jenkins job run (full build), without additional compile checks? Can it be done via just arc diff with some additional args?


If you wonder, why in the world we decided to use Differential to produce full builds - there is idea to eventually run check-build on literally anything, then store the produced artifacts somewhere, and later reuse them in build-windows and build-linux.


edit: it seems that --plan-changes won’t work, because Arcanist seemingly first creates the revision, and then changes its status to Changes planned - and this small window is enough for Herald to trigger the build…

#2

it seems that --plan-changes won’t work, because Arcanist seemingly first creates the revision, and then changes its status to Changes planned - and this small window is enough for Herald to trigger the build…

I think that the intended behavior is to not create-and-change, but create-with-status, specifically to avoid the behavior you’re describing; Are you fully up to date on arcanist and phabricator?


If I’m understanding you correctly, you have 2 forms of builds - short and long, and you want to have the short build run by default, unless the user specifically requests a long build, and your issue is that when they ask for a long build, you also get short running.

The easiest solution is to just let the 2 builds run, and add another node to your Jenkins (that’s what I did when I was in the same situation); The engineering time you’ll spend trying to optimize this is much more expensive then the machine time it will take to actually run.

There are no extra flags you can put on a arc diff that will help you; The only semi-reasonable options you have are

  1. putting some magic text in the revision description
  2. putting some magic file/content in the revision content
  3. using some naming convention in the local branch the dev is sending the revision from
  4. building some complex Jenkins-side system to figure out the dual build, and kill in in-flight/pre-flight.

Options 1 and 2 are not good because the message will remain later;
Option 3 is not good because (a) I’m not sure that information is even available to Herald, (b) it might not exist at all, depending on how users work, and © might conflict with whatever other branch naming conventions they use;
Option 4 requires engineering work and an extra database.

When I built such a solution, engineers would trigger the long build using the Revision page UI (I’ve added a button), which would normally happen a while after the short build was done anyway.


Running check-build on everything sounds totally good idea to me, based on the idea that running the build is cheap. Storing and reusing artifacts is a question of the eng/ops costs of keeping and finding the artifacts later, vs the time to re-build them when needed.

#3

We are, with phabricator maybe being one iteration behind.

You are correct in about short and long, but short still takes about 10-30 minutes and occupies almost all the nodes that can be used for long builds. That is exactly why I considered the question in the first place - because it is not a no-op, and the node time could be utilized better.

I valued the options you listed and indeed they all are not ideal. I believe we have some options on speeding up short builds so they would be less of a burden, so we will investigate this direction.

A workaround we also think to use instead of --plan-changes is to forbid pushing changes without accepted revision, and make short build trigger only when a reviewer is assigned to a revision. If this works, people would be able to create a “dummy” revisions simply for long builds triggered manually, and usual revisions with reviewers mentioned for short builds triggered automatically.

#4

…make short build trigger only when a reviewer is assigned to a revision…

That sounds reasonable to me; Also, have you tried the Draft mode (httpshttps://secure.phabricator.com/T13010)?

In any case, I suspect that you’ll have the short build re-triggered when a reviewer is added, even if the long build already completed successfully; If your build pipeline is limited, you might have to tackle this too.
I think I added a Priority plugin to Jenkins probably around this too - I made the long builds higher-priority, because they are always triggered manually.