Hackage and ghc shipped package contents mismatch

[Moritz Angermann]

Dear friends,

the other day I've run into an issue where I could not build a package.  It turned out that
the underlying reason was that the packages ghc ships with are not necessarily the same
as they are on hackage. Ryan was kind enough to open Ticket 16199[1].

My use case is that I reduce the set of packages I use from the package database that ghc
ships, and build the others myself.  Thus if packages in the package database as shipped
with ghc do not match the ones found on hackage for the SAME VERSION, I run into issues.

I've since devised a script to compute the difference between packages in GHC and the ones
on hackage with the same version[2]. 

I've then sat out to compute the differences for ghc8.4 and ghc8.6 using the following
approach:

for release in ghc-8.4.1-release ghc-8.4.2-release ghc-8.4.3-release ghc-8.4.4-release \
               ghc-8.6.4-release ghc-8.6.2-release ghc-8.6.3-release; do
  git checkout $release
  git reset --hard HEAD && git clean -xffd
  git submodule update --init --recursive && git clean -xffd
  ../verify-packages.sh
  mkdir -p ../$release
  find package-diffs -not -empty -type f -exec cp {} ../$release \;
done

I've compiled them[3] for inspection (and because I need to patch my hackage packages to
match the ones that ghc ships).

I'm sorry to report that I found discrepancies between the package we ship with GHC and
the ones on haskell for each ghc release in 8.4 and 8.6.  Some changes are minor, such
as version bumps, as can be represented with revisions on hackage as well.  Other are not
so minor. The one over which I tripped was transformers-0.5.5.0.  The diff[4] is 20K.

I therefore propose that we make sure to only ship packages with GHC that match their
respective versions on hackage.

Cheers,
 Moritz

--
[1]: http://ghc.haskell.org/trac/ghc/ticket/16199
[2]: https://gitlab.haskell.org/ghc/ghc/merge_requests/139
[3]: https://github.com/angerman/haskell.nix/tree/master/patches
[4]: https://github.com/angerman/haskell.nix/blob/master/patches/ghc863/transformers-0.5.5.0.patch