Testing GHC against Hackage via CI

[Richard Eisenberg]

How will this affect workflow of developers submitting patches? For example, suppose I write a user-facing change that breaks some of these packages. Am I expected to patch up the breakage? How? Will the CI infrastructure detect this before merging or after?

To be clear, I don't have specific answers I'm looking for here. In particular, I think it's reasonable to ask that the developer of a user-facing feature make sure that head.hackage works with the feature. This has the distinct advantage of making sure the developer knows that their patch breaks code and how it must be fixed. Indeed, this can all be fodder to force the developer to update the GHC migration guide alongside their work in fixing head.hackage. I just want to know what the steps are. :)

Thanks,
Richard

PS: And thanks, generally, to all of you doing the heavy lifting to get this infrastructure in place. This is all quite wonderful, even if there have been some bumps in the road en route!

> On Mar 5, 2019, at 1:10 PM, Ben Gamari <ben at well-typed.com> wrote:
> 
> Hello everyone,
> 
> As many of you know, recently I have been working on bringing up
> infrastructure for using CI-produced binary distributions to build a
> subset of Hackage using the head.hackage patchset.
> 
> Happily, this effort has now converged on a usable result, embodied in
> two merge requests:
> 
> * https://gitlab.haskell.org/ghc/head.hackage/merge_requests/2
> * https://gitlab.haskell.org/ghc/ghc/merge_requests/465
> 
> ghc/head.hackage!2 adds CI support to head.hackage. In addition to the
> usual pass/fail status, this job produces (e.g. [1]) a few additional
> products:
> 
> * a JSON summary of the run, describing the dependency graph and
>   pass/fail state of each package. we can feed to an external service to
>   track newly-failing packages.
> 
> * a (rendered) graphviz graph showing the dependency structure of the
>   built packages, each node colored by its pass/fail state
> 
> * a tarball of build logs, each including statistics from
>   -ddump-timings. The hope here is that we can use this to track
>   compiler performance on real-world code
> 
> ghc/ghc!465 adds a job to GHC's own CI pipeline to trigger a
> head.hackage job using the binary distribution produced earlier in the
> pipeline. This job will run automatically under a variety of
> circumstances:
> 
> * via the scheduled nightly pipeline
> 
> * on merge requests labelled with the `user-facing` label
> 
> * when manually started using a button on the MR's Pipelines tab
> 
> Currently the head.hackage MR tests only a small number of Hackage
> packages (and their dependencies). Specifically, we currently build
> `aeson`, `criterion`, `singletons`, `servant`, and `scotty`. This pulls
> in 127 packages in total.
> 
> There are a few things that remain to be done:
> 
> * Better document the existence of the job and how it is triggered
> 
> * Document how to update the list of tested packages
> 
> * Work out how to handle tracking of persistent breakage (e.g. we want
>   a notification when a package initially breaks but not in later
>   builds)
> 
> * Automatically push patched packages to a Hackage repository (e.g.
>   head.hackage.org) as a result of CI.
> 
> Cheers,
> 
> - Ben
> 
> 
> [1] https://gitlab.haskell.org/ghc/head.hackage/-/jobs/38022
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs