[GHC DevOps Group] GitLab migration status

Tue Dec 18 18:33:34 UTC 2018

Hey simon!
I actually did some digging. And it looks like currently gitlab basically
does a git diff of the two branches and does that computation on every page
reload.

1). There currently is no caching —- some of the folks on the Haskell infra
team could help perhaps but they need to be given access to the setup for
that to happen.

2) the complexity of the git diff is linear in the commit history length
between the two branches being compared

Some of these things could be mitigated easily, but I spoke with one of the
folks on the Haskell infra team and it sounds like ultimately either
patching the gitlab gitaly server or providing a more efficient api
compatible replacement are what would be needed.

On Mon, Dec 17, 2018 at 3:18 AM Simon Jakobi via ghc-devs <
ghc-devs at haskell.org> wrote:

> Hi Ben,
>
> in my experience Gitlab has been extremely slow at showing commit diffs to
> the point that it gives up and returns a 502:
> https://gitlab.staging.haskell.org/ghc/ghc/issues/15944
>
> Is this possibly related to any resource constraints on our instance?
>
> Cheers,
> Simon
>
> Am Mo., 17. Dez. 2018 um 06:29 Uhr schrieb Ben Gamari <ben at well-typed.com
> >:
>
>> TL;DR. Given somewhat slower-than-expected progress on the Trac import I
>>        suggest that we implement a pared-down migration on Tuesday.
>>        See "The Plan" below.
>>
>>
>> Hello everyone,
>>
>> Over the last few weeks we have been hard at work preparing the
>> migration to GitLab. Currently the following things are ready:
>>
>>  * Hosting of GHC's repositories and those of its mirrors have been
>>    prepared.
>>
>>  * Continuous integration has been configured for GHC.
>>
>>    All-in-all the GitLab migration has been quite timely since we were
>>    recently notified by CircleCI of billing changes which will soon make
>>    it quite difficult for us to continue using their services (see the
>>    thread on ghc-devops for details).
>>
>>    Thankfully, moving CI to GitLab has been mostly painless and has
>>    even enabled us to introduce testing of platforms which were
>>    previously inaccessible to us under CircleCI.
>>
>>  * The various linters which previously ran via `arc lint` and gitolite
>>    post-receive hooks have been ported to CI jobs.
>>
>>  * The Trac ticket migration is looking good although there are still a
>>    significant number of details which need to be sorted out.
>>
>>  * The Wiki migration is in a similar state.
>>
>> Over the past weeks we have been in constant contact with GitLab's FOSS
>> outreach group, who have been quite helpful in getting the eyes of
>> GitLab employees on the issues affecting our transition. Thanks to
>> especially to David Planella for his help so far.
>>
>> Unfortunately, there is one issue in particular [1] which is currently
>> blocking the Trac migration. From my discussions with GitLab's upstream
>> it sounds like it may be possible for them to prioritize a fix in the
>> short-term. However, our aggressive migration timeline is a fair bit
>> faster than GitLab's development cycle and consequently this certainly
>> won't happen before our planned migration on Tuesday.
>>
>>
>> # The Plan
>>
>> Given what remains to be done in the Trac migration I believe it would
>> be a mistake to move ahead with the full migration as planned. However,
>> in the interest of re-gaining functional continuous integration of
>> patches as soon as possible I propose that we move ahead with moving
>> code review on Tuesday.
>>
>> The plan would be as follows:
>>
>>  1. We setup the final gitlab.haskell.org instance tomorrow; since the
>>     Trac migration will not be run will need to create new accounts on
>>     instance.
>>
>>  2. We begin officially accepting merge requests on this fresh GitLab
>>     instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
>>     become GHC's official upstream repository.
>>
>>  3. We allow a week of transition time where new Differentials will
>>     continue to be accepted via Phabricator.
>>
>>  4. After this transition period we place Phabricator in read-only mode.
>>
>>  5. When we are confident in the Trac migration (likely after the new
>>     year) we move ahead with importing tickets and the wiki
>>
>> Previously I was skeptical of any plan that involved running the Trac
>> migration against a live GitLab instance. However, further reflection
>> I believe such a migration is safe and feasible. Moreover, given the
>> constraints set upon us by the impending CircleCI changes, I think this
>> is our best option to ensure continuity of CI.
>>
>> Thoughts?
>>
>> Cheers,
>>
>> - Ben
>>
>>
>> [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980
>> _______________________________________________
>> Ghc-devops-group mailing list
>> Ghc-devops-group at haskell.org
>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20181218/53b00567/attachment.html>