<div dir="ltr"><div>Congratulations, and well done. Changing basic infrastructure is a huge and thankless job.</div><div><br></div><div>Alan<br></div><br></div><br><div class="gmail_quote"><div dir="ltr">On Thu, 27 Dec 2018 at 08:27, Ben Gamari <<a href="mailto:ben@well-typed.com">ben@well-typed.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">TL;DR. <a href="https://gitlab.haskell.org/ghc/ghc.git" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc.git</a> is now the official<br>
upstream GHC repository. Various introductory notes are<br>
discussed. Let me know if you have any trouble.<br>
<br>
Also, please do verify the correctness of the email address<br>
associated with your Trac account in the next few weeks. It will<br>
be used to map users when we transition Trac tickets to GitLab.<br>
<br>
<br>
<br>
Hello everyone,<br>
<br>
I am happy to announce that CI on GHC's GitLab instance [1] is now<br>
stable. At this point <a href="https://gitlab.haskell.org/ghc/ghc.git" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc.git</a> is to be<br>
considered the official upstream repository of GHC.<br>
<br>
The rest of this email is meant to serve as a brief introduction and<br>
status update. It can also be viewed on the GitLab Wiki [2].<br>
<br>
[1] <a href="https://gitlab.haskell.org/" rel="noreferrer" target="_blank">https://gitlab.haskell.org/</a><br>
[2] <a href="https://gitlab.haskell.org/ghc/ghc/wikis/welcome" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc/wikis/welcome</a><br>
<br>
<br>
# Getting started<br>
<br>
To get started on GitLab you will first want to either create a new account<br>
[1] or login with your GitHub credentials [2].<br>
<br>
Once you have an account you should add an SSH key [3] so that you can push<br>
to your repositories. If you currently have commit rights to GHC notify me<br>
(Ben Gamari) of your user name so I can grant you similar rights in GitLab.<br>
<br>
<br>
[1] <a href="https://gitlab.haskell.org/users/sign_in" rel="noreferrer" target="_blank">https://gitlab.haskell.org/users/sign_in</a><br>
[2] <a href="https://gitlab.haskell.org/users/auth/github" rel="noreferrer" target="_blank">https://gitlab.haskell.org/users/auth/github</a><br>
[3] <a href="https://gitlab.haskell.org/profile/keys" rel="noreferrer" target="_blank">https://gitlab.haskell.org/profile/keys</a><br>
<br>
<br>
# Updating your development environment<br>
<br>
You can updated existing working directory (assuming the usual upstream<br>
remote name of `origin`) for the new upstream repository location by<br>
running the following:<br>
<br>
git remote set-url origin <a href="https://gitlab.haskell.org/ghc/ghc.git" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc.git</a><br>
git remote set-url --push origin git@gitlab.haskell.org:ghc/ghc<br>
<br>
This is all that should be necessary; a quick `git pull origin master`<br>
should verify that everything is working as expected.<br>
<br>
<br>
# Continuous integration<br>
<br>
Continuous integration is now provided by GitLab's native continuous<br>
integration infrastructure. We currently test a variety of<br>
configurations, including many that neither Phabricator nor<br>
CircleCI/Appveyor previously tested (see [1] for an example run):<br>
<br>
* With the make build system:<br>
* x86_64/Linux on Fedora 27, Debian 8, and Debian 9<br>
* i386/Linux on Debian 9<br>
* aarch64/Linux on Debian 9 (currently broken due to a variety of<br>
issues)<br>
* x86_64/Windows<br>
* x86_64/Darwin<br>
* x86_64/Linux on Debian 9 in a few special configurations:<br>
* unregisterised (still a bit fragile due to #16085)<br>
* integer-simple<br>
* building GHC with -fllvm<br>
* With Hadrian:<br>
* x86_64/Linux on Debian 9<br>
* x86_64/Windows (currently broken due to #15950)<br>
<br>
We also run a slightly larger set of jobs on a nightly basis. Note that<br>
binary distributions are saved from most builds and are available for<br>
download for a few weeks (we may put in place a longer retention policy<br>
for some builds in the future).<br>
<br>
There are admittedly a few kinks that we are still working out,<br>
particularly in the case of Windows (specifically the long build times<br>
seen on Windows). If you suspect you are seeing spurious build failures<br>
do let us know.<br>
<br>
To make the best use of our limited computational resources our CI<br>
builds occur in three stages:<br>
<br>
* lint: the style and correctness checkers which would previously be<br>
run by `arc lint` and `git push`<br>
<br>
* build: Debian 9 Linux x86_64 built with make and Hadrian<br>
<br>
* full-build: the remaining configurations<br>
<br>
If a build fails at an earlier phase no further phases will be run.<br>
<br>
<br>
[1] <a href="https://gitlab.haskell.org/ghc/ghc/pipelines/568" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc/pipelines/568</a><br>
<br>
<br>
# Structuring your merge request<br>
<br>
With the transition to GitLab GHC is moving to a model similar to that<br>
used by GitHub. If you have a Differential on Phabricator we will finish<br>
review there. However, please post new patches as merge requests on<br>
GitLab.<br>
<br>
Note that Phabricator and GitLab have quite different models for<br>
handling patches. Under Phabricator a Differential is a single patch<br>
with no further structure; larger changes can be composed of multiple<br>
dependent Differentials.<br>
<br>
Under GitLab's model a merge request is a git branch consisting of<br>
one or more patches. Larger changes can be handled in one of two ways:<br>
<br>
a. a set of dependent merge requests, each of which to be squashed when<br>
merged.<br>
<br>
b. a single branch with each atomic change made in a single, buildable<br>
commit<br>
<br>
Due to the difficulty of maintaining dependent merge requests, I would<br>
recommend that contributors making larger changes use method (b).<br>
<br>
<br>
# Submitting your merge request for review<br>
<br>
Depending upon whether you have push rights to the GHC repository there<br>
are two ways to submit a merge request:<br>
<br>
* if you have push access you can push a branch directly to<br>
git@gitlab.haskell.org:ghc/ghc.git and open merge request.<br>
<br>
In this case please do follow the usual branch naming conventions:<br>
<br>
* prefix all branch names with `wip/`<br>
<br>
* if you are fixing a particular ticket consider using the name<br>
`wip/TNNNN`<br>
<br>
* if not you can create a fork using the "Fork" button on the project<br>
page [1] and push your branch there<br>
<br>
In either case after you have pushed your branch open a merge request<br>
against ghc/ghc [2].<br>
<br>
[1] <a href="https://gitlab.haskell.org/ghc/ghc/forks/new" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc/forks/new</a><br>
[2] <a href="https://gitlab.haskell.org/ghc/ghc/merge_requests/new" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc/merge_requests/new</a><br>
<br>
<br>
# Reviewing and merging merge requests<br>
<br>
As always, all contributors are encouraged to help review proposed<br>
changes. If you are unfamiliar with GitLab's review interface please see<br>
GitLab's user documentation [1]. Here are a few quick highlights for<br>
those who are familiar with GitHub but haven't yet used GitLab:<br>
<br>
* As with GitHub, GitLab supports both inline and out-of-line comments.<br>
<br>
* Comments that are actionable (known as "discussions") can be marked<br>
as resolved and collapsed.<br>
<br>
* Comments can be left on both changed and unchanged lines<br>
<br>
* Revisions of a merge request can be viewed and compared using the<br>
two drop-down menus at the top of the Changes tab<br>
<br>
* Merge requests can require approvals from particular users before<br>
considered as mergable<br>
<br>
* Merge requests can be placed in "merge when CI passes" state, which<br>
will cause merge requests to be merged as soon as they are green<br>
<br>
>From this point onward all changes to GHC will be merged via<br>
GitLab's merge requests facility and must pass CI before being merged.<br>
To ensure that GHC's git history remains linear ghc/ghc will use GitLab's<br>
"fast-forward without a merge commit" merge strategy. Consequently you<br>
will be asked to rebase merge requests which are not fast-forward merges<br>
before merging (a convenient "Rebase" button will appear if the rebase<br>
can be carried out without conflicts.<br>
<br>
[1] <a href="https://gitlab.com/help/user/discussions/index.md#discussions" rel="noreferrer" target="_blank">https://gitlab.com/help/user/discussions/index.md#discussions</a><br>
<br>
<br>
# Status of the Trac migration<br>
<br>
Tobias will be continuing work on the Trac ticket migration after a bit<br>
of a holiday break. Hopefully by mid-January we will be able to move<br>
forward on this part of the migration; I will share more details about<br>
this as they develop.<br>
<br>
In the meantime, users of Trac should check and possibly update the<br>
email address associated with their account [1]. This address will be<br>
used to correlate Trac users with their GitLab equivalents so the<br>
correctness of this address will be important in preserving attribution<br>
information during the Trac import.<br>
<br>
[1] <a href="https://ghc.haskell.org/trac/ghc/prefs" rel="noreferrer" target="_blank">https://ghc.haskell.org/trac/ghc/prefs</a><br>
<br>
<br>
# Next steps<br>
<br>
We are actively working on cleaning up a few remaining issues with CI:<br>
<br>
* build times are still very long on Windows, despite the fact that we<br>
are only building the `quick` build flavour on that platform;<br>
consequently GitLab CI Windows builds do sometimes timeout<br>
when we are faced with long build queues.<br>
<br>
* we at times run low on disk space on our Windows builder runners,<br>
resulting in occasional spurious build failures<br>
<br>
* Appveyor builds (which are supposed to supplement the native GitLab<br>
builds) rarely seem to finish<br>
<br>
GitLab upstream has been incredibly supportive of our transition effort<br>
and has expressed interest in assisting us with issues that we<br>
encounter. Our current requests can be found on our migration effort's<br>
tracking ticket [1]. If you find any additional bugs or workflows that<br>
could be improved please do let me know and I can raise the matter with<br>
GitLab.<br>
<br>
[1] <a href="https://gitlab.com/gitlab-org/gitlab-ce/issues/55039" rel="noreferrer" target="_blank">https://gitlab.com/gitlab-org/gitlab-ce/issues/55039</a><br>
<br>
<br>
# Acknowledgments<br>
<br>
I would like to acknowledge several parties for their contributions to<br>
this effort:<br>
<br>
* Packet.net and Google X for their generous donation of hosting for<br>
continuous integration and web hosting<br>
<br>
* GitLab and their Open Source program for many productive discussions,<br>
their generous support, and the GitLab Ultimate license used by<br>
<a href="http://gitlab.haskell.org" rel="noreferrer" target="_blank">gitlab.haskell.org</a>.<br>
<br>
* Davean Scies for his help procuring the hosting services that power<br>
our continuous integration.<br>
<br>
* Tweag.io for their offer of help and advice<br>
<br>
* Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work<br>
in setting up the new instance, sorting out the details of the<br>
migration, and debugging problems when they arose<br>
<br>
Finally, thanks to GHC's contributors for their patience during this<br>
transition; it has been a long process which has stolen a significant<br>
amount of attention from other matters. My apologies we have been a bit<br>
less responsive than usual in code review and ticket triage over the<br>
past month or two. Regardless, I am hopeful that this wait will be<br>
worthwhile.<br>
<br>
<br>
# Final thoughts<br>
<br>
This is not only a milestone for the GitLab migration but also for GHC<br>
itself. For the first time GHC has fully-automated testing, proposed<br>
patch CI, and release generation across the full range of Tier 1<br>
configurations it supports, with passing builds in all cases.<br>
<br>
We are very excited to begin this next chapter of GHC's development and<br>
are looking forward to your feedback on how we can further improve our<br>
new infrastructure. Onward and upwards!<br>
<br>
Cheers,<br>
<br>
- Ben<br>
_______________________________________________<br>
Ghc-devops-group mailing list<br>
<a href="mailto:Ghc-devops-group@haskell.org" target="_blank">Ghc-devops-group@haskell.org</a><br>
<a href="https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group" rel="noreferrer" target="_blank">https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group</a><br>
</blockquote></div>