Phabricator for patches and code review

Tue Jun 24 17:12:37 UTC 2014

Richard,

Thanks, these are all actually really excellent questions.

> 1) I'm just setting things up on my machine. It says to `arc install-certificate` in my GHC directory. Is it important precisely which clone of GHC my directory is set up against? For example, my "pull" origin is git://git.haskell.org/ghc.git and my "push" origin is ssh://git@git.haskell.org/ghc.git. Is this the right set-up? If this bit matters, could you add it to the page? Or, if not, could you comment on what `arc` is pulling from the ghc directory?

The way 'arc install-certificate' knows what URL to use is based on a
file in the GHC repository, called .arcconfig - this tells arcanist
where the Phabricator instance is, and what the project callsign is.
GHC's copy is here: https://github.com/ghc/ghc/blob/master/.arcconfig

So all you need to do is run 'install-certificate' inside the main GHC
repo, and you're done! If you've got a copy of HEAD from the past two
weeks or so, you should be golden. Once install-certificate is done,
it will write the certificate into a file called ~/.arcrc, so you
don't have to install it again.

> 2) I'm confused about what, precisely, `arc diff` does. You describe that it updates the review available online. Does that reference some git commits? Do I need to push by `wip` branch before `arc diff`ing? Do I *never* need to push my branch, because `arc diff` pushes it for me? Do I *never* need to push my branch, because Phab uses other ways of moving the code around? For better or worse, I tend to keep my branches local until I'm ready to merge with master, and I want to know if this needs to change.

You _never_ have to push a branch upstream if you don't want to (but
if you'd like to keep your changes in a safe place until you merge
them, feel free to put them on a wip/* branch!)

What 'arc diff' does is this: when you run it, it calculates which
commits you have made. It does this by comparing your current git
repository to that of the *upstream* repository, the GHC repo. When it
calculates the set of commits, 'arc diff' then sends them all for
review. When you make a new commit, and say 'arc diff' again, it
updates it.

> 3) You say "A change cannot be merged until at least one reviewer has signed off." Does this mean "merged with ghc-7.8" (or whatever the current stable branch is)? Or does it mean "merged with master"? The former is the status quo, but with a new route, so to speak. The latter is something new, as several of us push directly to `master` without a review. I'm not against such a change, per se, just trying to understand it.

Not quite. All changes go into master first, after it has been
reviewed. Afterwords, they may be picked onto the stable branch by me
or Herbert.

My only statement is that we cannot merge a patch from Phab to master
until at least one person has given it the OK. If you put a review on
Phabricator, I believe you should at least wait for one person to
review it. That is, after all, what Phabricator is for! :)

But note: you are *never* required to use Phabricator for your
changes! If your workflow is good for you, I don't want to interrupt
it. Instead, I want you to submit reviews so other people can read and
understand your work - existing committers *are not* forced to do
this. For example Richard, you obviously keep in close touch with
Simon, so he's likely aware of your changes and their needs. There's
no reason for you to ask SPJ for a review on a change you already have
a handle on - just go ahead and push it!

> 4) Is it now compulsory to use Phab when contributing? This page makes it sound that way. Again, no complaints -- just trying to understand it.

Nope! It might be so later (for external contributors without commit
access), but I first need to document all the workflows so
contributors can get a handle on it. So thanks for being the guinea
pig :) But it won't ever be compulsory for all committers.

Again, I want people to use Phab so they can have others read,
understand, and comment about their work. There's no hard-line rule on
if you should or should not submit a review - I will totally trust
your judgement as to what you think is your best workflow.

> 5) The page says to add `austin` as a reviewer. I would expect `thoughtpolice`. What's up with Phab usernames? Do other people I know and love have Phab usernames distinct from Trac usernames?

Sorry - I'm the only outlier. I believe every other person has the
same username except for me. :P (This is partly due to the fact we use
Phabricator for other Haskell.org projects as well, and we didn't
explicitly plan to use it for GHC initially - so 'austin' was the name
I picked when first setting up the instance).

> 6) Who is the reviewer `#ghc`? Is this related to the IRC channel? What does the # signify?

Good question I totally forgot about. In Phabricator, there is a
notion of 'projects', which have members. There is a project known as
'GHC', and in Phab, you can refer to this group of people within the
project by a hashtag, essentially. So `#ghc` means "All people who are
in the GHC project". The # is just a way of referring to a project or
group, basically.

Here's the current listing of people in the GHC project:
https://phabricator.haskell.org/project/view/2/ - you should be able
to (and feel free to) add yourself to that group from that linked
page.

> 7) I'm baffled by "Landing reviews". The `arc land` bit is clearer, but I'm still unsure of what my local state and the git upstream state must be beforehand for this to work. Will this ask me for a commit message? Will it use the one I specified to Phab during `arc diff`? In general, I'm confused about how much info `arc` pulls from various places to do its work. I know I could learn by doing, but I'm afraid of mashing on the Phab and/or git history as I do so.

This is a good question, but it's one probably best answered with an
example. I'll work something into the page to make this easier to
understand.

> 8) Some of the same questions surround `arc patch`: What does my git state need to be for this to work?

Ditto with this one - I'll work it into a tangible example to make it clearer.

9) How do I contribute to others' revisions? Or, will this be obvious
what it comes to pass?

Normally, you just do things 'the git way', so don't let Phab get in
your way here. If you want to contribute to someone elses patches -
work with them and send them your patch! Then they can update the
review with you and their own work included.

However, there is *another* related notion in Phab which is that of
"commandeering" a review. Which should be pretty self explanatory: you
take it over, and now *you* are in control of submitting new diffs.
Sometimes this may happen if I work on something, leave it rotting,
and someone else may want to pick it up.

Anyway, thanks for all the excellent questions Richard, these were
things that definitely would have confused others, so I'm glad you
asked. I'll update the wiki page to be clearer.

On Tue, Jun 24, 2014 at 9:09 AM, Richard Eisenberg <eir at cis.upenn.edu> wrote:
> Thanks so much for writing this! I have some questions:
>
> 1) I'm just setting things up on my machine. It says to `arc install-certificate` in my GHC directory. Is it important precisely which clone of GHC my directory is set up against? For example, my "pull" origin is git://git.haskell.org/ghc.git and my "push" origin is ssh://git@git.haskell.org/ghc.git. Is this the right set-up? If this bit matters, could you add it to the page? Or, if not, could you comment on what `arc` is pulling from the ghc directory?
>
> 2) I'm confused about what, precisely, `arc diff` does. You describe that it updates the review available online. Does that reference some git commits? Do I need to push by `wip` branch before `arc diff`ing? Do I *never* need to push my branch, because `arc diff` pushes it for me? Do I *never* need to push my branch, because Phab uses other ways of moving the code around? For better or worse, I tend to keep my branches local until I'm ready to merge with master, and I want to know if this needs to change.
>
> 3) You say "A change cannot be merged until at least one reviewer has signed off." Does this mean "merged with ghc-7.8" (or whatever the current stable branch is)? Or does it mean "merged with master"? The former is the status quo, but with a new route, so to speak. The latter is something new, as several of us push directly to `master` without a review. I'm not against such a change, per se, just trying to understand it.
>
> 4) Is it now compulsory to use Phab when contributing? This page makes it sound that way. Again, no complaints -- just trying to understand it.
>
> 5) The page says to add `austin` as a reviewer. I would expect `thoughtpolice`. What's up with Phab usernames? Do other people I know and love have Phab usernames distinct from Trac usernames?
>
> 6) Who is the reviewer `#ghc`? Is this related to the IRC channel? What does the # signify?
>
> 7) I'm baffled by "Landing reviews". The `arc land` bit is clearer, but I'm still unsure of what my local state and the git upstream state must be beforehand for this to work. Will this ask me for a commit message? Will it use the one I specified to Phab during `arc diff`? In general, I'm confused about how much info `arc` pulls from various places to do its work. I know I could learn by doing, but I'm afraid of mashing on the Phab and/or git history as I do so.
>
> 8) Some of the same questions surround `arc patch`: What does my git state need to be for this to work?
>
> 9) How do I contribute to others' revisions? Or, will this be obvious what it comes to pass?
>
> I realize I've asked a lot here, and it might be too much to expect all of these answers to be on the page. If that's the case, could you perhaps link to relevant manuals or places to learn more? The way `arc` works, in particular, seems like magic; magic is very powerful, but it can be equally dangerous, and so I'd like to learn more.
>
> Thanks so much for doing all this!
> Richard
>
> On Jun 23, 2014, at 10:44 AM, Austin Seipp <austin at well-typed.com> wrote:
>
>> Hi all,
>>
>> I went ahead and took some time to write some stuff down about
>> Phabricator, including some basic tips on the workflows and
>> applications here:
>>
>> https://ghc.haskell.org/trac/ghc/wiki/Phabricator
>>
>> It's definitely going to need more expanding. Do let me know if
>> anything is confusing.
>>
>> On Wed, Jun 18, 2014 at 2:38 AM, Jan Stolarek <jan.stolarek at p.lodz.pl> wrote:
>>> Duh, ignore what I wrote. I just realized I'm working on a non-rebased branch :-)
>>>
>>> Janek
>>>
>>> Dnia środa, 18 czerwca 2014, Jan Stolarek napisał:
>>>> I read the friendly Arcanist manual and I wonder if we intend to have a
>>>> default .arcconfig file in the GHC repo? From the docs it seems like a good
>>>> idea.
>>>>
>>>> Janek
>>>>
>>>> Dnia wtorek, 17 czerwca 2014, Simon Marlow napisał:
>>>>> On 13/06/14 10:47, Jan Stolarek wrote:
>>>>>> It seems that most people are in favour of using Phabricator for code
>>>>>> review. So what are the next steps? Can we just start using the
>>>>>> existing phabricator instance? I'm working on some code right now that
>>>>>> definitely needs reviewing.
>>>>>
>>>>> You can use it, and a few of us have already been doing so.  There isn't
>>>>> any Trac integration yet, but it works nicely for patch review.
>>>>>
>>>>> There's a short intro doc here:
>>>>> https://secure.phabricator.com/book/phabricator/article/differential/,
>>>>> but it's not hard to figure out the basics, and you'll learn by watching
>>>>> how other people use it.  If you go to the Herald tool you have yourself
>>>>> automatically subscribed to diffs that touch areas of the code that
>>>>> you're interested in.
>>>>>
>>>>> Pro tip: the keyboard shortcuts are really useful, especially "z".  Hit
>>>>> "?" to see all the shortcuts.
>>>>>
>>>>> Cheers,
>>>>> Simon
>>>>>
>>>>>> Janek
>>>>>>
>>>>>> Dnia niedziela, 8 czerwca 2014, Simon Marlow napisał:
>>>>>>> On 07/06/2014 07:21, Manuel M T Chakravarty wrote:
>>>>>>>> So, why not put everything on GutHub and use pull requests and so on?
>>>>>>>
>>>>>>> github just isn't great for doing code reviews. No side-by-side diffs,
>>>>>>> and it sends you a separate email for every single comment, there's no
>>>>>>> concept of a "review" consisting of multiple inline comments (unless
>>>>>>> I've missed something). I'm afraid if we were using this for regular
>>>>>>> reviews I would have to disable the email notifications, which makes
>>>>>>> it significantly less useful.
>>>>>>>
>>>>>>>> SimonM writes that Phabricator is better than GitHub. I’m happy to
>>>>>>>> believe that, but he also writes that using it requires installing
>>>>>>>> local software and quite a bit of work. Moreover, I like to add that
>>>>>>>> lots of people already know how to use GitHub and probably few know
>>>>>>>> Phabricator.
>>>>>>>>
>>>>>>>> So, we are talking about having a somewhat better tool in return for
>>>>>>>> three very significant disadvantages: (1) local installation, (2)
>>>>>>>> work to set up and maintain Phabricator, and (3) effort by many
>>>>>>>> people to learn to use it.
>>>>>>>
>>>>>>> Well, you've tipped the balance with "somewhat" and "significant"
>>>>>>> here, I'd say Phabricator is "significantly" better than github for
>>>>>>> code reviews, while installing arc is "somewhat" annoying :-)
>>>>>>>
>>>>>>> I have to admit it's not a no-brainer, but I do worry that github just
>>>>>>> wouldn't cut it for doing a lot of code reviewing, whereas I spend my
>>>>>>> life inside Phabricator so I know it works really well.
>>>>>>>
>>>>>>> What's more, github doesn't let you put animated gifs in code reviews.
>>>>>>> Need I say more?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Simon
>>>>>>>
>>>>>>>> We also have a constant lack of sufficient men power. So, why spend
>>>>>>>> effort on building our own infrastructure, which will only increase
>>>>>>>> the hurdle for contributors (as they have to deal with an unknown
>>>>>>>> system)? Let’s outsource the effort to GitHub.
>>>>>>>>
>>>>>>>> Manuel
>>>>>>>>
>>>>>>>> Simon Peyton Jones <simonpj at microsoft.com>:
>>>>>>>>> At the moment GHC's main sources aren't on github, which means that
>>>>>>>>> that (in my highly imperfect understanding) people can't submit pull
>>>>>>>>> requests or use their code review mechanisms.  Moreover, most people
>>>>>>>>> don't have commit rights on the main GHC server, so if someone wants
>>>>>>>>> to offer a patch they can really only do so in textual form attached
>>>>>>>>> to Trac. People with commit rights can make a branch, but there's a
>>>>>>>>> danger that over a decade we'll accumulate zillions of dead branches
>>>>>>>>> which people forgot to delete.  I think on github the branch is in a
>>>>>>>>> different repo, belonging to the patch author.
>>>>>>>>>
>>>>>>>>> So we really don't have a good work flow for creating, reviewing,
>>>>>>>>> modifying, and finally apply patches.  I am no expert on these
>>>>>>>>> matters. If Phabricator would help with that I'm all for it.  But
>>>>>>>>> perhaps there are other alternatives?  Or is Phab the lead thing.
>>>>>>>>> Will it stay around?
>>>>>>>>>
>>>>>>>>> Also before going too far I'd really like someone to document the
>>>>>>>>> workflow carefully, and make sure it works from Windows equally
>>>>>>>>> well.
>>>>>>>>>
>>>>>>>>> I'm not too stressed out about losing the review trail of a patch.
>>>>>>>>> Much of it will be commenting on stuff that no longer appears in the
>>>>>>>>> final patch.  Anything that's important should appear in a Note in
>>>>>>>>> the source code; even the commit messages are invisible until you
>>>>>>>>> really start digging.
>>>>>>>>>
>>>>>>>>> Simon
>>>>>>>>>
>>>>>>>>> | -----Original Message-----
>>>>>>>>> | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
>>>>>>>>> | Austin Seipp
>>>>>>>>> | Sent: 06 June 2014 05:06
>>>>>>>>> | To: ghc-devs at haskell.org
>>>>>>>>> | Subject: RFC: Phabricator for patches and code review
>>>>>>>>> |
>>>>>>>>> | Hello all,
>>>>>>>>> |
>>>>>>>>> | Recently, while doing server maintenance, several of the
>>>>>>>>> | administrators for Haskell.org set up an instance of
>>>>>>>>> | Phabricator[1], located at https://phabricator.haskell.org
>>>>>>>>> |
>>>>>>>>> | For those who aren't aware, Phabricator (or "Phab") is a suite of
>>>>>>>>> | tools for software development. Think of it like a polished,
>>>>>>>>> | semi-private GitHub with a lot of applications and tools for all
>>>>>>>>> | kinds of needs. We've been using it to do issue tracking for
>>>>>>>>> | Haskell.org maintenance and like it a lot so far.
>>>>>>>>> |
>>>>>>>>> | One very nice aspect of Phabricator though is it has a very nice
>>>>>>>>> | code review tool, called 'Differential', that is very useful. For
>>>>>>>>> | people who have used a tool like Review Board, it's similar.
>>>>>>>>> | Furthermore, it has a very convenient userland tool called
>>>>>>>>> | 'Arcanist' which makes it easy for newcomers to post a review and
>>>>>>>>> | get it merged when it's ready all from the command line.
>>>>>>>>> |
>>>>>>>>> | I'd like to see if people are interested in using Phab _strictly_
>>>>>>>>> | for code review of GHC patches. It is a dedicated tool
>>>>>>>>> | specifically for this, and I think it works much better than Trac
>>>>>>>>> | or inline GitHub comments.
>>>>>>>>> |
>>>>>>>>> | Also, Phab can also support post-commit reviews. So if I touch
>>>>>>>>> | something in the runtime system and just push, perhaps Simon or
>>>>>>>>> | Edward would like to look, and they can be alerted right when I do
>>>>>>>>> | this, and then yell if I did something stupid.
>>>>>>>>> |
>>>>>>>>> | Before I go much further, I'd like to ask: is there *any* interest
>>>>>>>>> | in this? Or are people satisifed with Trac? The primary
>>>>>>>>> | motivations are roughly, in no particular order:
>>>>>>>>> |
>>>>>>>>> |  1) Code review is good for everyone, a good way for people to
>>>>>>>>> | learn the code and ask questions, and useful to give feedback to
>>>>>>>>> | newcomers. And even experienced GHC hackers can learn things from
>>>>>>>>> | reading code, as we all do regularly, or find things that need
>>>>>>>>> | cleanup.
>>>>>>>>> |
>>>>>>>>> |  2) Phabricator in particular makes it very easy to submit patches
>>>>>>>>> | for review. To submit a patch, I just run the command 'arc diff'
>>>>>>>>> | and it Does The Right Thing. It also makes it easy to ensure
>>>>>>>>> | people are *alerted* when a patch might be relevant to them.
>>>>>>>>> |
>>>>>>>>> |  3) They can be uploaded and created from the command line, and
>>>>>>>>> | merged easily afterwords the same way. This is particularly useful
>>>>>>>>> | for newcomers, and for me. :)
>>>>>>>>> |
>>>>>>>>> |  4) Differential is dedicated to code review, and much better at
>>>>>>>>> | it than just reading patches on Trac IMO.
>>>>>>>>> |
>>>>>>>>> |  5) It supports both post-commit code review, as well as
>>>>>>>>> | pre-commit review. Post commit would be especially useful for us
>>>>>>>>> | too, I think.
>>>>>>>>> |
>>>>>>>>> | Point #2 and #3 are mostly relevant for me, because I mostly
>>>>>>>>> | handle incoming patches. But I think in general it would be nice,
>>>>>>>>> | and make it a lot easier for newcomers to submit patches, and us
>>>>>>>>> | to look over them.
>>>>>>>>> |
>>>>>>>>> | Here's an example of a Differential code review:
>>>>>>>>> |
>>>>>>>>> | https://phabricator.haskell.org/D4
>>>>>>>>> |
>>>>>>>>> | This is a demo using my 'wip/ermsb' patch. You'll need to create
>>>>>>>>> | an account to login, but it shouldn't be much trouble, you can
>>>>>>>>> | login several ways. I'll fix the login requirement soon. Feel free
>>>>>>>>> | to read the code, comment on it, and play around. It's more of a
>>>>>>>>> | demonstration, but real code review would be welcome too. :)
>>>>>>>>> |
>>>>>>>>> | If people are interested in doing this, I can add notes to the
>>>>>>>>> | wiki pages for newcomers, and I'll send another email about Phab
>>>>>>>>> | so people can understand it a little better. But I want to ask
>>>>>>>>> | first.
>>>>>>>>> |
>>>>>>>>> | There is an argument that our team is so small, code review has
>>>>>>>>> | unnecessary burdens. But I think Phab could help a lot with
>>>>>>>>> | tracking outside patches and getting good reviews for incoming
>>>>>>>>> | patches, and it'll make it easier for newcomers. And experienced
>>>>>>>>> | pros can probably learn a thing as well.
>>>>>>>>> |
>>>>>>>>> | Again, to be clear, I don't propose we migrate anything to
>>>>>>>>> | Phabricator from, say, Trac. There's no real pressure to do so and
>>>>>>>>> | it would be tons of work. I only propose we use it for code
>>>>>>>>> | review, which is perfectly fine, and how other projects like LLVM
>>>>>>>>> | do code review (they use Bugzilla).
>>>>>>>>> |
>>>>>>>>> | I also don't think the usage of Phabricator should be mandatory
>>>>>>>>> | (unless we decide that later because we like it), but I would like
>>>>>>>>> | to see people use it if possible.
>>>>>>>>> |
>>>>>>>>> | [1] http://phabricator.org
>>>>>>>>> |
>>>>>>>>> | --
>>>>>>>>> | Regards,
>>>>>>>>> |
>>>>>>>>> | Austin Seipp, Haskell Consultant
>>>>>>>>> | Well-Typed LLP, http://www.well-typed.com/
>>>>>>>>> | _______________________________________________
>>>>>>>>> | ghc-devs mailing list
>>>>>>>>> | ghc-devs at haskell.org
>>>>>>>>> | http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ghc-devs mailing list
>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ghc-devs mailing list
>>>>>>>> ghc-devs at haskell.org
>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ghc-devs mailing list
>>>>>>> ghc-devs at haskell.org
>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>
>>>>>> _______________________________________________
>>>>>> ghc-devs mailing list
>>>>>> ghc-devs at haskell.org
>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>
>>>> _______________________________________________
>>>> ghc-devs mailing list
>>>> ghc-devs at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>
>>>
>>
>>
>>
>> --
>> Regards,
>>
>> Austin Seipp, Haskell Consultant
>> Well-Typed LLP, http://www.well-typed.com/
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>

-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/