Trac to Phabricator (Maniphest) migration prototype

[Ben Gamari]

Matthew Pickering <matthewtpickering at gmail.com> writes:

> Dear devs,
>
Hi Matthew and Dan,

First, thanks for your work on this; it is an impressive effort.
Reconstructing a decade of tickets with broken markup, tricky syntax,
and a strange data model is no easy task. Good work so far!

On the whole I am pleasantly surprised by how nicely Maniphest seems to
hang together. I have pasted my notes from my own reflection on the pros
and cons of both systems below. On re-reading them, it seems clear that
Trac does leave us with a number of issues which Phabricator may
resolve.

As I've expressed in the past, I think we should consider preservation
of ticket numbers to be an absolute requirement of any migration. To do
otherwise imposes a small cost on a large number of people for a
very long time with little benefit. GHC infrastructure exists to
support GHC, not the other way around.

However, ticket numbers notwithstanding I think I would be fine moving
in this direction if the community agrees that this is the direction we
want to go in. 

There are a few questions that remain outstanding, however:


What do others think of this?
=============================

Does Phabricator address the concerns that others, including those
outside of the usual GHC development community, have raised about our
issue tracking in the past? It would be interesting to hear some outside
voices.


How do we handle the stable branch?
===================================

One important role that the issue tracker plays is ensuring that 
patches are backported to the stable branch when appropriate. In Trac we
handle this with a "merge" state (indicating that the ticket has been
fixed in `master`, but the fix needs to be backported) and the milestone
field (to indicate which release we want to backport to).

A significant amount of GHC's maintenance load is spent backporting and
updating tickets, so it's important that this process works smoothly and
reliably. I think this is may be an area where Phabricator could
improve the status quo since the workflow currently looks something like
this,

 1. Someone merges a patch to `master` fixing an issue; if the commit
    message mentions the ticket number then our infrastructure
    automatically leaves a comment referencing the commit on the ticket.

 2. Someone (usually the committer) places the ticket in `merge` state
    and sets the milestone appropriately

 3. I merge the patch to the stable branch

 4. I close the ticket and manually leave a comment mentioning the SHA
    of the backported commit.

In particular (4) is more work than it needs to be; ideally comment
generation would be automated as it is for commits to `master` but
Trac's comment-on-commit functionality is a bit limited, so this is
currently not an option.

I'm not sure what Phabricator's analogous workflow to the above might
look like. It seems that Phabricator's Releeph module may be in part
intended for this use-case, but it seems to have some unfortunate
limitations (e.g. the inflexibility in branch naming) that make it hard
to imagine it being usable in our case.

Setting aside Releeph, perhaps the best solution would be to continue
with our current workflow: we would retain the "status" state and
milestone projects would take the place of the current "milestone"
field. If I'm not mistaken Phabricator can be configured to mention
commits on stable branches in the ticket history, so this should help
point (4).


Which fields should be preserved?
=================================

Our Trac instance associates a lot of structured metadata with each
ticket. I generally think that this is a good thing, especially compared
to the everything-is-a-tag model which I have found can quickly become
unmaintainable. Unfortunately, Trac makes our users pay for these fields
with a confusing ticket form.

It appears that Phabricator's Transactions [1] module may allow us to have
our cake and eat it too: we can define one form for users to create
tickets and another, more complete form for developer use. In light of
this I don't see why we would need to fall back to the
everything-is-a-tag. Matthew, what did you feel was
less-than-satisfactory about the proper-fields approach? I fear that
relevant metadata like GHC version, operating system and architecture
simply won't be provided unless the user is explicitly prompted; I
personally find the cue to be quite helpful. Presumably contributors can
set Herald rules for notification on these fields if they so desire.

In the particular case of the "Component" field, I personally try to
set this correctly when possible and have certainly found it useful as a
search criterion. However, I suspect it would be fine as a tag. I also
know that Simon PJ is quite fond of the test case field (although
few others as as diligent in keeping this one up to date).


How would we migrate and what will become of Trac?
==================================================

The mechanics of migration will take time and effort to work out. If we
decide this is the right direction I think we should be cautious in
setting timelines; we should take as much time as we need to do it
correctly. Regardless, we should gather a consensus on the general
direction before we start hashing this out.


Thanks again for your effort on this, Matthew and Dan, and sorry it took
me so long to finally get these notes out.

Cheers,

- Ben


[1] https://secure.phabricator.com/book/phabricator/article/forms/



Notes
=====

These were largely for my sake to keep track of the pros and cons of the
two options. I've nevertheless included them here for completeness.

What does Maniphest do well?
----------------------------

 * Actively developed: Phabricator will continue to improve in the
   future.

 * Metadata: Custom fields are supported.

 * Flexible user interface: Custom fields can be hidden from the new
   ticket form to prevent user confusion.

 * Familiarity: Many users may feel more at home in Phabricator's interface;
   reMarkup's similarity to Markdown also helps.

 * Integration: Having Phabricator handle code review, release
   management, and issue tracking will hopefully reduce maintenance
   workload.

 * Notifications: Herald's rule-based notifications are quite handy.


What does Maniphest do poorly?
------------------------------

 * Flexibility of search: The search options feel a bit more limiting
   than Trac; in particular the ability to show arbitrary columns in
   search results seems conspicuously missing.

 * Legibility: This is admittedly to some extent a matter of aesthetics
   but the search results list feels very busy and is quite difficult to
   quickly scan. This is made exacerbated by the fact that some aspects
   of the the color scheme are quite low contrast (e.g. grey against
   white for closed tickets). This hurts quite a bit since a number of
   contributors spend a significant amount of time looking through lists
   of tickets. Perhaps we could convince the Phacility people to provide
   a more legible, compact presentation option.


What does Trac do well?
-----------------------

 * Convenient cross-referencing: while the syntax is a bit odd, once you
   acclimate it is quite liberating to be able to precisely
   cross-reference tickets, wiki documents, and comments without copying
   links around.

 * Automation of ticket lifecycle: Trac tickets progress through their
   lifecycle (e.g. from "new" to "patch" to "merge" to "closed"
   statuses) through predefined actions. This means that moving a ticket
   through its lifecycle typically only requires one click and the right
   thing happens with no additional effort. I think this is a great model,
   although in practice it's not clear how much we benefit from it
   compared to a typical Maniphest workflow.

 * Rich metadata: Tickets can have a variety of metadata fields
   which can be queried in sophisticated ways .


What does Trac do poorly?
-------------------------

 * Familiarity: Many users feel rather lost in Trac

 * Active development: Trac is largely a stagnant project.

 * Spam management: Keeping spam at bay has been a challenge. We seem to
   have it under control at the moment, but I wonder how long this will
   last for.

 * Safety: I have personally lost probably a half-dozen hours of my life
   to Trac eating comments.

 * Integration with code review workflow: We use Phabricator for CI and
   code review; the only thing that remains on Trac are our tickets and
   the Wiki. Keeping these two resources in sync is time-consuming and
   error-prone.

 * Full text search: Trac's full text search capabilities are generally
   terrible. While I've tried working around this using PostgreSQL's
   native full text search functionality, the result is poorly
   integrated into Trac and consequently hard to use.

 * Customizability of the ticket form: While the rich metadata that Trac
   supports can be very helpful developers, it can also be confusing to
   users. Ideally we would hide these fields but Trac does not give us
   the ability to do so.

 * Relations between tickets: Trac has essentially no first-class notion
   of ticket relatedness. Even duplicate tickets need to be manually
   annotated in both directions.

 * Keywords are hard to discover and apply

 * Fine-grained notification support is nearly non-existent

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170108/e3c05577/attachment-0001.sig>