Release Management - Phabricator & Jenkins

Question
#1

Hi all -

We have been using Phabricator for a few years to track dev tasks and host Git repos.

  • We use Arcanist to submit code and review output from our devs
  • We use Jenkins separately to build Docker images / NPM packages based on Phabricator repos.
  • Builds in Jenkins are triggered manually and versioning is currently managed there via a simple “last version” file on the Jenkins side.
  • We run a Project in Phabricator for Release Management with a milestone per release. Here, we manually track Jenkins build numbers for our various packages by typing them in (:frowning:) and manually compile a summary of included changes/fixes (:frowning: again).

I would like to automate versioning/releases within Phabricator and integrate properly with Jenkins and its output, but cannot find any best practice docs or recommendations on how to achieve this.

I know we can trigger builds from Phabricator using a Jenkins plugin/Harbormaster/Herald Rules (so the build integration part should be fine).

There is a release management feature “Releeph” which I see in the experimental apps section, but it looks like that has been discontinued?

Has anyone got any pointers they can offer on how to do release management and how to put Phabricator in control of versioning (semantic versioning), with effective Jenkins integration?

Thanks in advance - any clarification required, please shout.

#2

It would depend a lot on what your specific goals actually are, and is probably outside of the Phabricator specific realm

In any case, here is what we do

Every one of our repos has a custom .lcconfig in it (our naming convention). This json file contains various things, one of those things is the current semantic version number

We have a custom set of wrapper tools. One of those is to do release builds - So we do lc build --increment-version which

  • Increments the version number in .lcconfig
  • Git Tags the commit with the new version number
  • Calculates a list of every task / revision that has been landed since the last version number increase
  • Writes all of that to a commit message, commits it
  • Pushes back to Phabricator
  • Herald rule tags that with a custom Build project
  • Harbormaster does things by telling Jenkins to run tests, reports, etc. because we have a Build commit.

From this we do various other things, but you get the idea. A specialized commit contains the build information

The nice thing about this is

  • The build commit contains a build log of every change
  • Every task/revision that has something new in that build get’s a “mention” so we can see that Build x.y.z referenced Task Txyz or Revision Dxyz

Here is an example from our system

Release Build Commit

image
image656×635 29.4 KB

And then

image

Not sure if this is helpful to anyone, but it is our workflow.

One thing to remember that with us - Anything that needs to go to production is committed to master with arc land

2 Likes
#3

Hey @ekubischta -

Massive thanks for the write up, that’s really helpful. Storing the last version in each repo makes sense for us and it is a great idea to have an optional “increment” build flag so we only do that when we do a release build.

Our product currently has an umbrella release number tied to a Milestone in Phabricator, which includes a number of constituent sub-packages of different versions. The sub-package versions can currently be incremented separately to the Product. I need to work out whether to keep doing it like this (within each package, have a “product version” and a “me version” number which we track & increment), or whether it would be worth changing the version to be uniform across everything… which could introduce its own difficulties.

Product version: 2.5.7
Package A - 1.2.3
Package B - 2.3.4
Package C - 3.4.5

Any thoughts or recommendations on that front would be brilliant.

#4

The Releaph app is planned to be replaced with the “Release” app, but all that exists right now is a bunch of text at https://secure.phabricator.com/T9530 (Oh, there’s some code! forgot about that!).
The plan doesn’t cover “automatic versioning” so much as it creates a mutable “Release” object that you can iterate over - looks like it might be similar to your Milestone-is-version setup, or to @ekubischta’s Build Commit

Personally, I think having an Umbrella/Product version that is distinct from the sub-product/package’s versions is the sanest way to go - you can communicate the parent version to users that use the whole thing, and sub-components internally for dev. Trying to keep them all in sync would get you in a small, specialized hell of re-releasing unchanged products.
I’d might even go a step further and remove the “Product version” tag from each package, in the name of decoupling.

#5

Thanks for your response @avivey - I’ll certainly keep my eye on that app, looks like it’s stalled at the moment so won’t be around any time soon?

Thanks also for your comment on Product vs Component versioning; that makes sense - keep them separate and don’t expose the component versions to the users. We’ll have to think about dependencies/version compatibility - that can be based on component versions.

@ekubischta - quick question on your methodology; could you please confirm how you determine which tasks have been implemented since the last commit? Is it based on the last commit date, or do you have some sort of task tagging and pick up any which have been closed without a tag?

Cheers,
Matt

#6

Since every commit to master is in the form prescribed by phab each commit looks like this

commit xyxyxyxyx
Author: name <name@place.com>
Date:   Thu Dec 10 18:41:28 2020 +0000

Did a cool thing with some cool code

Maniphest Tasks: T12345

Differential Revision: https://url.to.phab.com/D6789

So, when running lc build --increment-version the tool gets a list of all the commits between the last tagged version commit and HEAD (my new build) and then I parse through the commit messages and find all of the revisions and tasks - I then write those out to my new build commit message.

We have a suite of cli tools that we wrote that accomplishes this - So in our world, we also have a tool like :

lc build-log which just shows me the list of tasks and revisions since the last tagged build - Or, lc build-log 1.0.12 which shows me a list of all tasks and revisions since the specified tag

(Also, maybe importantly, in our workflow, ALL revisions are linked to at least 1 task)