Relocating (some of) GHC's core-libraries to github.com/haskell

[Herbert Valerio Riedel]

On 2014-04-29 at 12:27:38 +0200, Simon Marlow wrote:

[...]

>>> I think that's fine.  As Simon points out, we already have lagging
>>> repo functionality in the form of the submodule links, so the repo on
>>> git.haskell.org can be a pure mirror.
>>
>> Just so I get this right, does "pure mirror" here mean that we don't
>> want users to be able to push to the automatically mirrored repo on
>> git.haskell.org at all, but rather the only way to get any commits into
>> the git.haskell.org mirrored repo would be push it via the GitHub repo?
>>
>> (I'd like that, as it would make the set-up easier and hopefully less
>> confusing, as there'd be only a single data-flow path)
>
> Makes sense to me, but how does that interact with your post-commit
> hook that checks for validity of the submodule updates?

(btw, it's actually a pre-receive/update script, as a post-commit hook
runs to late to be able to reject ref-updates)


The submod-referential-check script would still work, as it would check
that at least at the time a push to ghc.git was done, the respective
submodule commits were present at git.haskell.org

I've asked the GitHub admins to disallow force-pushes on the new repos I
created at github.com/haskell/ as a safety measure (they don't support
disallowing branch-deletion though, so there's still a way to force-push
by workarounding via branch deletion+recreation, but I'd trust the users
we give write-access to[1] to not abuse this loophole)

Moreover, I'd configure the sub-repos at git.haskell.org to never prune
unreachable objects automatically as a short-term measure. This would
allow to manually recover any "lost" commits, and make them reachable
again, even if github.com/haskell already forgot about them.

As for the mirroring itself, this may lag a little bit at first, until I
get the scripting better: right now git.haskell.org would poll
github.com every 10 minutes or so for new commits. Later-on that would
be improved by enabling a post-commit webhook on github to notify
git.haskell.org about new commits to reduce the mirroring latency. I
hope this is enough for now.



Also, I'd like to share the automatic mirroring-workflow with other
packages already living at github.com/haskell, such as containers or
bytestring.

Long story short, the workflow for modifying a core-lib package formerly
living at git.haskell.org would be to

 1.) get the commit somehow into the new upstream at github.com[2]

 2.) wait a little bit till the commit gets propagated automatically to
     git.haskell.org

 3.) commit and push the submodule gitlink ref update in ghc.git to
     git.haskell.org



Cheers,
   hvr


 [1]: I've created a 'GHC developers' team in the github.com/haskell org
      some time ago, which mirrors the users who have write access to
      git.haskell.org; I'd simply assign those to the new repos for
      now. So there should be no authorization-regressions.


 [2]: We'll provide a convenient way to redirect pushes to
      git.haskell.org/packages/$PKG to its real writable upstream repo

      I recently learned about Git's

       url.<base>.insteadOf
       url.<base>.pushInsteadOf

      redirection feature, you'd only have to set the redirect rules
      once, and then you could 'git clone --recursive' simply using the
      canonical http://git.haskell.org URLs, but it would instead go to
      github.com for fetch/pushes as instructed. This may be far easier
      and more robust than having sync-all rewriting the pushurls.

      this would also make it easy, to temporarily switch to github.com,
      if git.h.o is down, w/o having to reconfigure all your ghc source
      trees, as those rules are a setting that can live in ~/.gitconfig