Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))

Herbert Valerio Riedel hvriedel at
Mon Jul 24 10:24:21 UTC 2017

Hi everyone,

As part of the recent GHC 8.2.1 release and the associated boot-library
publishing to Hackage, I picked up again experimenting with a
reinstallable lib:ghc package primarily for the benefit of uploading
haddocks for lib:ghc (w/ `--hyperlinked-source`) to Hackage:

However, this also provides somewhat of a tech-preview of a
reinstallable lib:ghc package, which would help mitigate a big issue we
had in the past with lib:ghc and its dependencies being frozen to
specific versions, and which in combination with cabal's nix-style
pkg store that finally provide us with the ability to have multiple
instances of the same library versions co-existing in the same package
db, is no longer a requirement. So we could have lib:ghc more easily pick
dependencies up (but still within reason!) like e.g. lib:text without
forcing everyone to the single lib:text version bundled with GHC. And
we'd also gain the ability to publish patch-level releases to lib:ghc
only inbetween proper minor GHC releases (e.g. to support new versions
of library dependencies).

Consider this artificial example demo package which requires a different
version of binary than the one bundled with GHC 8.2.1:

--8<---------------cut here---------------start------------->8---
-- ghc-demo.cabal
name:                ghc-demo
version:             0
build-type:          Simple
cabal-version:       >=2.0

executable ghc-demo
  main-is:             Main.hs
  build-depends:       base ^>= 4.10, ghc ^>= 8.2.1, binary ==
  default-language:    Haskell2010
--8<---------------cut here---------------end--------------->8---

Trying to build this will have the cabal solver complain that the
installed instance of ghc requires a different (& installed) version of
binary, i.e. binary-

Moreover, the source version of lib:ghc on Hackage has the manual
`buildable` cabal flag which defaults to false, and currently prevents
lib:ghc to be reinstalled automatically (by enabling an unsatisfiable

--8<---------------cut here---------------start------------->8---
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: base- (dependency of ghc-demo-0)
next goal: ghc (dependency of ghc-demo-0)
rejecting: ghc-8.2.1/installed-8.2... (conflict: binary==, ghc => binary==
trying: ghc-8.2.1
rejecting: ghc-8.2.1:-buildable (conflict: base==, ghc -buildable => base<0)
rejecting: ghc-8.2.1:+buildable (manual flag can only be changed explicitly)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-demo, binary, base, ghc, ghc-8.2.1:buildable
--8<---------------cut here---------------end--------------->8---

However, if we unlock this by toggling the flag (NB: this only works on
linux/x86_64; it's explained later why), e.g.

--8<---------------cut here---------------start------------->8---
-- cabal.project file
packages: .

package ghc
  flags: +buildable
--8<---------------cut here---------------end--------------->8---

The cabal solver would come up with an install-plan:

--8<---------------cut here---------------start------------->8---
Resolving dependencies...
Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
 - binary- {binary-} (lib) (requires download & build)
 - ghc-boot-8.2.1 {ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb} (lib) (requires build)
 - ghci-8.2.1 {ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1} (lib) (requires build)
 - ghc-8.2.1 {ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c} (lib) +buildable (requires build)
 - ghc-demo-0 {ghc-demo-0-inplace-ghc-demo} (exe:ghc-demo) (first run)

Configuring  binary- (lib)
Building     binary- (lib)
Installing   binary- (lib)
Finished     binary- (lib)
Configuring  ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Building     ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Installing   ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Finished     ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Configuring  ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Building     ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Installing   ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Finished     ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Configuring  ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Building     ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Installing   ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Finished     ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)

Configuring executable 'ghc-demo' for ghc-demo-0..
Preprocessing executable 'ghc-demo' for ghc-demo-0..
Building executable 'ghc-demo' for ghc-demo-0..
[1 of 1] Compiling Main             ( Main.hs, /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo-tmp/Main.o )
Linking /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo ...
--8<---------------cut here---------------end--------------->8---

So, cabal was able to succesfully produce an executable which uses a
reinstalled lib:ghc w/ a different lib:binary version!

But now for the limitations: this currently works at best with a GHC
8.2.1 installation whose configuration matches the generated files I
manually included in the modified lib:ghc package in

which as you can see from the generated files (see e.g. Config.hs) was a
Linux/x86_64/libgmp/internal-libffi/... configuration.

...and this finally brings me to the purpose of why I wrote this email:
In order to make lib:ghc properly reinstallable we'd need to either

 a) have a way to regenerate the files autogen/* via a custom Setup.hs
    (by the likes of genprimcode or similiar)

or the lazy option,

 b) simply have those files installed in a place we can easily locate
    (again, from a custom Setup.hs)

Does this make any sense; which option do you prefer?

Also, I'd like to know if you can think of reasons why or situations
when the reinstalled lib:ghc wouldn't work; or other reasons why this is
a bad idea.


On 2016-06-05 at 19:02:29 +0200, Herbert Valerio Riedel wrote:
>> As an ultimate goal for a project like this, it would be nice if you could
>> do something like upgrade your base package but continue to use the ghc
>> package without incompatible types.
> That's one way to approach this. However, I've been experimenting with a
> more flexible way: Thanks to cabal-1.24's new nix-based tech-preview
> which gets rid of that `--force-reinstall` abomination for good, the
> prospect of simply recompiling the `ghc` cabal package becomes more
> attractive.
> This would get rid of the primary reason which currently constraints us
> to using the package versions bundled in GHC (like e.g. for
> `bytestring`) as soon as `ghc` enters an install-plan.
> I'm still investigating this, but early experiments were promising, but
> I need to reorganise GHC's source-tree a little bit to know for sure if
> this is feasable. But if this as I hope is possible, this could open up
> a lot of new possibilities!

More information about the ghc-devs mailing list