Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))
Herbert Valerio Riedel
hvriedel at gmail.com
Mon Jul 24 10:24:21 UTC 2017
Hi everyone,
As part of the recent GHC 8.2.1 release and the associated boot-library
publishing to Hackage, I picked up again experimenting with a
reinstallable lib:ghc package primarily for the benefit of uploading
haddocks for lib:ghc (w/ `--hyperlinked-source`) to Hackage:
http://hackage.haskell.org/package/ghc
However, this also provides somewhat of a tech-preview of a
reinstallable lib:ghc package, which would help mitigate a big issue we
had in the past with lib:ghc and its dependencies being frozen to
specific versions, and which in combination with cabal's nix-style
pkg store that finally provide us with the ability to have multiple
instances of the same library versions co-existing in the same package
db, is no longer a requirement. So we could have lib:ghc more easily pick
dependencies up (but still within reason!) like e.g. lib:text without
forcing everyone to the single lib:text version bundled with GHC. And
we'd also gain the ability to publish patch-level releases to lib:ghc
only inbetween proper minor GHC releases (e.g. to support new versions
of library dependencies).
Consider this artificial example demo package which requires a different
version of binary than the one bundled with GHC 8.2.1:
--8<---------------cut here---------------start------------->8---
-- ghc-demo.cabal
name: ghc-demo
version: 0
build-type: Simple
cabal-version: >=2.0
executable ghc-demo
main-is: Main.hs
build-depends: base ^>= 4.10, ghc ^>= 8.2.1, binary == 0.8.5.0
default-language: Haskell2010
--8<---------------cut here---------------end--------------->8---
Trying to build this will have the cabal solver complain that the
installed instance of ghc requires a different (& installed) version of
binary, i.e. binary-0.8.5.1.
Moreover, the source version of lib:ghc on Hackage has the manual
`buildable` cabal flag which defaults to false, and currently prevents
lib:ghc to be reinstalled automatically (by enabling an unsatisfiable
constraint):
--8<---------------cut here---------------start------------->8---
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: base-4.10.0.0/installed-4.1... (dependency of ghc-demo-0)
next goal: ghc (dependency of ghc-demo-0)
rejecting: ghc-8.2.1/installed-8.2... (conflict: binary==0.8.5.0, ghc => binary==0.8.5.1/installed-0.8...)
trying: ghc-8.2.1
rejecting: ghc-8.2.1:-buildable (conflict: base==4.10.0.0/installed-4.1..., ghc -buildable => base<0)
rejecting: ghc-8.2.1:+buildable (manual flag can only be changed explicitly)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-demo, binary, base, ghc, ghc-8.2.1:buildable
--8<---------------cut here---------------end--------------->8---
However, if we unlock this by toggling the flag (NB: this only works on
linux/x86_64; it's explained later why), e.g.
--8<---------------cut here---------------start------------->8---
-- cabal.project file
packages: .
package ghc
flags: +buildable
--8<---------------cut here---------------end--------------->8---
The cabal solver would come up with an install-plan:
--8<---------------cut here---------------start------------->8---
Resolving dependencies...
Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
- binary-0.8.5.0 {binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac} (lib) (requires download & build)
- ghc-boot-8.2.1 {ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb} (lib) (requires build)
- ghci-8.2.1 {ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1} (lib) (requires build)
- ghc-8.2.1 {ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c} (lib) +buildable (requires build)
- ghc-demo-0 {ghc-demo-0-inplace-ghc-demo} (exe:ghc-demo) (first run)
Configuring binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Building binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Installing binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Finished binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Configuring ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Building ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Installing ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Finished ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Configuring ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Building ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Installing ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Finished ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Configuring ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Building ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Installing ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Finished ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Configuring executable 'ghc-demo' for ghc-demo-0..
Preprocessing executable 'ghc-demo' for ghc-demo-0..
Building executable 'ghc-demo' for ghc-demo-0..
[1 of 1] Compiling Main ( Main.hs, /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo-tmp/Main.o )
Linking /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo ...
--8<---------------cut here---------------end--------------->8---
So, cabal was able to succesfully produce an executable which uses a
reinstalled lib:ghc w/ a different lib:binary version!
But now for the limitations: this currently works at best with a GHC
8.2.1 installation whose configuration matches the generated files I
manually included in the modified lib:ghc package in
http://hackage.haskell.org/package/ghc-8.2.1/src/autogen/
which as you can see from the generated files (see e.g. Config.hs) was a
Linux/x86_64/libgmp/internal-libffi/... configuration.
...and this finally brings me to the purpose of why I wrote this email:
In order to make lib:ghc properly reinstallable we'd need to either
a) have a way to regenerate the files autogen/* via a custom Setup.hs
(by the likes of genprimcode or similiar)
or the lazy option,
b) simply have those files installed in a place we can easily locate
(again, from a custom Setup.hs)
Does this make any sense; which option do you prefer?
Also, I'd like to know if you can think of reasons why or situations
when the reinstalled lib:ghc wouldn't work; or other reasons why this is
a bad idea.
Cheers,
hvr
On 2016-06-05 at 19:02:29 +0200, Herbert Valerio Riedel wrote:
>> As an ultimate goal for a project like this, it would be nice if you could
>> do something like upgrade your base package but continue to use the ghc
>> package without incompatible types.
>
> That's one way to approach this. However, I've been experimenting with a
> more flexible way: Thanks to cabal-1.24's new nix-based tech-preview
> which gets rid of that `--force-reinstall` abomination for good, the
> prospect of simply recompiling the `ghc` cabal package becomes more
> attractive.
>
> This would get rid of the primary reason which currently constraints us
> to using the package versions bundled in GHC (like e.g. for
> `bytestring`) as soon as `ghc` enters an install-plan.
>
> I'm still investigating this, but early experiments were promising, but
> I need to reorganise GHC's source-tree a little bit to know for sure if
> this is feasable. But if this as I hope is possible, this could open up
> a lot of new possibilities!
More information about the ghc-devs
mailing list