Any ways to test a GHC build against large set of packages (including test suites)?

[Ben Gamari]

On August 10, 2018 7:55:38 AM EDT, "Ömer Sinan Ağacan" <omeragacan at gmail.com> wrote:
>I also briefly looked at hackage.head. As far as I understand it
>doesn't
>out-of-the-box provide a way to build a large set of packages, right?
>It'd be
>useful if I had a package that I want to test against GHC HEAD but
>currently it
>doesn't help me, unless I'm missing something.
>
>Ömer
>
>Ömer Sinan Ağacan <omeragacan at gmail.com>, 10 Ağu 2018 Cum, 11:39
>tarihinde şunu yazdı:
>>
>> Hi,
>>
>> This is working great, I just generated my first report. One problem
>is stm-2.4
>> doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not
>published on
>> Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if
>there's
>> anything that can be done about this. Apparently stm blocks 82
>packages (I
>> don't know if that's counting transitively or just packages that are
>directly
>> blocked by stm). Any ideas about this?
>>
>> Ömer
>>
>> Ömer Sinan Ağacan <omeragacan at gmail.com>, 9 Ağu 2018 Per, 14:45
>> tarihinde şunu yazdı:
>> >
>> > Ah, I now realize that that command is supposed to print that
>output. I'll
>> > continue following the steps and keep you updated if I get stuck
>again.
>> >
>> > Ömer
>> >
>> > Ömer Sinan Ağacan <omeragacan at gmail.com>, 9 Ağu 2018 Per, 13:20
>> > tarihinde şunu yazdı:
>> > >
>> > > Hi Manuel,
>> > >
>> > > I'm trying stackage-head. I'm following the steps for the
>scheduled build in
>> > > .circleci/config.yml. So far steps I took:
>> > >
>> > > - Installed ghc-head (from [1]) to ~/ghc-head
>> > > - Installed stackage-build-plan, stackage-curator and
>stackage-head (with
>> > >   -fdev) from git repos, using stack.
>> > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
>> > > - curl
>https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
>> > > --output metadata.json
>> > > - curl
>https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
>> > > --output $BUILD_PLAN.yaml
>> > >
>> > > Now I'm doing
>> > >
>> > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
>> > > --ghc-metadata metadata.json --outdir build-reports
>> > >
>> > > but it's failing with
>> > >
>> > >     The combination of target and commit is new to me
>> > >
>> > > Any ideas what I'm doing wrong?
>> > >
>> > > Thanks
>> > >
>> > > [1]:
>https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
>> > >
>> > > Ömer
>> > >
>> > > Ömer Sinan Ağacan <omeragacan at gmail.com>, 7 Ağu 2018 Sal, 23:28
>> > > tarihinde şunu yazdı:
>> > > >
>> > > > Thanks for both suggestions. I'll try both and see which one
>works better.
>> > > >
>> > > > Ömer
>> > > >
>> > > > Manuel M T Chakravarty <chak at justtesting.org>, 7 Ağu 2018 Sal,
>18:15
>> > > > tarihinde şunu yazdı:
>> > > > >
>> > > > > Hi Ömer,
>> > > > >
>> > > > > This is exactly the motivation for the Stackage HEAD works
>that we have pushed at Tweag I/O in the context of the GHC DevOps
>group. Have a look at
>> > > > >
>> > > > >   https://github.com/tweag/stackage-head
>> > > > >
>> > > > > and also the blog post from when the first version went live:
>> > > > >
>> > > > >  
>https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
>> > > > >
>> > > > > Cheers,
>> > > > > Manuel
>> > > > >
>> > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan
><omeragacan at gmail.com>:
>> > > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I'd like to test some GHC builds + some compile and runtime
>flag combinations
>> > > > > > against a large set of packages by building them and
>running test suites. For
>> > > > > > this I need
>> > > > > >
>> > > > > > - A set of packages that are known to work with latest GHC
>> > > > > > - A way to build them and run their test suites (if I could
>specify compile and
>> > > > > >  runtime flags that'd be even better)
>> > > > > >
>> > > > > > I think stackage can serve as (1) but I don't know how to
>do (2). Can anyone
>> > > > > > point me to the right direction? I vaguely remember some
>nix-based solution for
>> > > > > > this that was being discussed on the IRC channel, but can't
>recall any details.
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > > Ömer
>> > > > > > _______________________________________________
>> > > > > > ghc-devs mailing list
>> > > > > > ghc-devs at haskell.org
>> > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > > > >
>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Head.hackage doesn't have an out of the box package set but it is quite straightforward to construct such a set. While I now tend to use nix, in the past I generally just constructed a dummy cabal package listing the packages of interest as dependencies.

There are two approaches to choosing a set of packages: extract the packages from Stackage's build-constraints.yaml or just additively build up a set from the top of your head. I find the latter is often more realistic; stackage is now large enough that even getting a fraction of it to build with a prerelease compiler can be a significant undertaking.