[GHC DevOps Group] The future of Phabricator

[Ben Gamari]

TL;DR. For several reasons I think we should consider alternatives to
       Phabricator. My view is that GitLab seems like the best option.


Hello everyone,

Over the past year I have been growing increasingly weary of our
continued dependence on Phabricator. Without a doubt, its code review
interface is the best I have used. However, for a myriad of reasons I
am recently questioning whether it is still the best tool for GHC's
needs.


# The problem

There are a number of reasons why I am currently uncertain about
Phabricator.

For one, at this point we have no options for support in the event that
something goes wrong as the company responsible for Phabricator,
Phacility, has closed their support channels to non-paying customers.
Furthermore, in the past year or two Phacility has been placing their
development resources in the parts their customers pay them for, which
appear to be much different that the parts that we actively use. For
this reason, some parts that we rely on seem oddly half-finished.

This concern was recently underscored by some rather unfortunate
misfeatures in Harbormaster which resulted in broken CI after the
Hadrian merge and now apparent bugs which have made it difficult to
migrate to the CircleCI integration we previously prepared.

Perhaps most importantly, in our recent development priorities survey
our use of Phabricator was the most common complaint by a fair margin,
both in case of respondents who have contributed patches and those who
have not. On the whole, past contributors and potential future
contributors alike have strongly indicated that they want a more
Git-like experience. Of course, this wasn't terribly surprising; this
is just the most recent case where contributors have made this
preference known.

Frankly, in a sense it is hard to blame them. The fact that users need
to install a PHP tool, Arcanist, to contribute anything but
documentation patches has always seemed like unnecessary friction to me
and I would be quite happy to be rid of it. Indeed we have had a quite
healthy number of GitHub documentation patches since we started
accepting them. This makes me thing that there may indeed be potential
contributoes that we are leaving on the table.


# What to do

With Rackspace support ending at the end of year, now may be a good
time to consider whether we really want to continue on this road.
Phabricator is great at code review but I am less and less certain that
it is worth the maintenance uncertainty and potential lost contributors
that it costs.

Moreover, good alternatives seem closer at-hand than they were when we
deployed Phabricator.


## Move to GitHub

When people complain about our infrastructure, they often use GitHub as
the example of what they would like to see. However, realistically I
have a hard time seeing GitHub as a viable option. Its feature set is simply
insufficient enough to handle the needs of a larger project like GHC
without significant external tooling (as seen in the case of Rust-lang).

The concrete reasons have been well-documented in previous discussions
but, to summarize,

 * its review functionality is extremely painful to use with larger
   patches

 * rebased patches lead to extreme confusion and lost review comments

 * it lacks support for pre-receive hooks, which serve as a last line of
   defense to prevent unintended submodule changes

 * its inability to deal with external tickets is problematic

 * there is essentially no possibility that we could eventually migrate
   GHC's tickets to GitHub's ticket tracker without considerable data
   loss (much less manage the ticket load that would result), meaning
   that we would forever be stuck with maintaining Trac.

 * on a personal note, its search functionality has often left me
   empty-handed

On the whole, these issues seem pretty hard to surmount.


## Move to GitLab

In using GitLab for another project over the past months I have been
positively surprised by its quality. It handles rebased merge requests
far better than GitHub, has functional search, and quite a usable review
interface. Furthermore, upstream has been extremely responsive to
suggestions for improvement [1]. Even out-of-the-box it seems to be
flexible enough to accommodate our needs, supporting integration with
external issue trackers, having reasonable release management features,
and support for code owners to automate review triaging (replacing much
of the functionality of Phabricator's Herald).

Finally, other FOSS projects' [3] recent migrations from Phabrictor to
GitLab have shown that GitLab-the-company is quite willing to offer help
when needed. I took some time this weekend to try setting up a quick GHC
instance [2] to play around with. Even after just a few hours of playing
around I think the result is surprisingly usable.

Out of curiosity I also played around with importing some tickets from
Trac (building on Matt Pickering's Trac-to-Maniphest migration tool).
With relatively little effort I was even able to get nearly all of our
tickets (as of 9 months ago) imported while preserving ticket numbers
(although there are naturally a few wrinkles that would need to be
worked out). Naturally, I think we should treat the question of ticket
tracker migration as an orthogonal one to code review, but it is good to
know that this is possible.


## Continue with Phabricator

Continuing with Phabricator is of course an option. Its review
functionality is great and it has served us reasonably well. However,
compared to GitLab and even GitHub of today its features seem less
distinguished than they once did. Moreover, the prospect of having to
maintain a largely stagnant product with no support strikes me as a
slightly dangerous game to play. Working around the issues we have
recently encountered has already cost a non-negligible amount of time.


# The bottom line

If it wasn't clear already, I think that we should strongly consider a
move to GitLab. At this point it seems clear that it isn't going to
vanish, has a steady pace of development, is featureful, and available.

However, these are just my thoughts. What do you think?

Cheers,

- Ben


[1] 11.4 will ship with a file tree view in the code review interface,
    which I reported
    (https://gitlab.com/gitlab-org/gitlab-ce/issues/46474) as being is
    one of the Phabricator features I missed the most during review

[2] https://gitlab.home.smart-cactus.org/ghc/ghc/issues/14641

[3] The GNOME and freedesktop.org projects have recently migrated, the
    former from a hodge-podge of self-hosted services and the latter
    from Phabricator
    
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20181030/9267cc20/attachment.sig>