Centralized Arcanist for consistency and CI

I’ve attempted to centralize Arcanist [diff] on a server to make it easier to integrate with CI and review processes. My question is: Is there anything being considered or worked on to help support making code reviews more automated. Having a server-side Arcanist for purposes of arc diff (haven’t tried other subcommands) seems so close to working seamlessly.

Conduit endpoint documentation seems to be limited to those who are well-versed in the underlying source code or those who succeed in trial-and-error. Phabricator as a whole seems to encourage user-interaction over command-line tools based on prompting and reading through issues. The flexibility to interactively choose is sometimes nice, but sometimes a strict contract of I will accept X and output Y, otherwise throw enumerated error Z (and thus being able to handle scenario Z) can be better.

My attempt at centralized Arcanist:

For first-time setup:

  • Create bot user to generate differentials from CI
  • Generate API token (because it’s a bot user no CLI token is available, though differences are not in docs)
  • Initialize a Git repository on the same server as the Phabricator-controlled bare repositories
  • Configure a standard set of lint tools per repository (centralization makes it easier to update and add rules globally)

For each arc diff:

  • Use CI [Jenkins] to run a pipeline that validates various things prior to entering review and triggers additional changes after arc diff generates (like status changes, emails, unit tests, etc.).
  • Pull information from ticket/issue/bug management system, including title, summary, reviewer(s), subscriber(s), etc. Send this information to a file to be referenced as a message-file.
  • Since the bot user will be the author, pull CI logged-in user as CC/Subscriber.
  • Perform VCS actions [Git] fetch, checkout branch so that checked-out branch is referenced by Arcanist
  • Execute arc diff to generate Differential with a --message-file including ticket information.

Issues:

  • Cannot automatically “commandeer to”/specify author as a bot
  • Cannot update a commandeered revision as a bot
  • Commandeered revisions move author bot to reviewer
  • Policies cannot be specified and differentials in general do not default to policies of the linked repository
  • No way to have multiple authors and Subscriber/CC has actions of a reviewer (and becomes a reviewer if dropdown action selected)

Versions in play:
$ arc --version
arcanist 5eda40337bb4135ca4929617602686302edc7cc0 (4 Aug 2017)
libphutil dfced13a45f376e017465e805e151b42ea7dd295 (27 Aug 2017)
phabricator b4cbea901845087f8903bdcd210303d7e6eace50 (Aug 29 2017)

I’m not sure I understand what you’re trying to do.

is it

I want to run some simple checks before creating a revision

If so, look at Lint and Unit Test support in arcanist.

is it:

I want to run CI on each new revision

?
If so, check out Harbormaster.

Is this it:

I want to run some CI on new revisions before calling humans in to review the change

?

If so, look at https://secure.phabricator.com/T2543 and https://secure.phabricator.com/T13010 which is building up to this.

If it’s something else, please explain what exactly you mean by “making code review more automated”.

The process I’m focusing on is creating an arcanist revision in a server-side configuration; the rest is just context for why I’m doing what I’m doing. There doesn’t seem to be a built-in way to allow bot users to specify an author or update a revision as a bot (a workaround could be to have multiple authors, but this is not supported). Also, when revisions are created via CLI, there isn’t a way to inherit the linked repository’s policies. I’d like to see the power to do certain actions via command line at my own risk if I’ve authenticated onto the server as a service account user.

Those are my primary concerns. The checks I need to run concern security reviews, architect reviews, documentation checks, static analysis, etc. belong in Git hooks, ticket system, and Jenkins.

I know you’re building support for your sister applications in Phabricator, but it would be nice if there could be a pass-through option to where a conforming CI of choice would be called and the CI would call Phabricator/Harbormaster when success or fail

I’m still not sure why you’re trying to use a server to create the revision, but this might help:

You can use arc --conduit-token <token> to impersonate a user, so that the bot will create the diff while pretending to be the human user behind it. Each user can create tokens in the Settings app.

I’m not sure what “pass-through” option means here. Harbormaster has the ability to notify a revision about CI results - eg, https://secure.phabricator.com/D18756#229387, and the expectation is that Harbormaster initiates the CI from a revision, and the CI notifies HM about results.

Setting policy

You can use arc call-conduit differential.revision.edit to edit most of the information of a revision, including edit policies - see https://secure.phabricator.com/conduit/method/differential.revision.edit/ for details. The only way to change Author value is to Commandeer a revision, which sets the authorship to the current user.
You could add a transaction type to set the author to any value, and use this transaction from the conduit method.

the power to do certain actions via command line at my own risk if I’ve authenticated onto the server as a service account user

You can use policies to determine who can do what. In some places, the action you’re trying to perform doesn’t have a defined policy (Or is not written), so you’d have to write your own.

The concept of “some users can bypass policies” is one that the upstream is very much against, because it allows attackers who can impersonate said user to escalate their attack and impersonate other users, and so on. Limiting this to specific actions would require maintaining a very long list of configuration options, along with a matching list of special-case handling in the code, which is a maintenance nightmare. If this is what you’re after, you can just make local changes in your phabricator code to match.