Relocatable dist

Wed Oct 25 01:43:34 UTC 2017

Hi,

just to elaborate a bit.  We *can* build cross compiler, and cross compile ghc with
the current make based build system.  I'm not certain how well packaging binary
distributions of cross compiled ghc's work.  I've only run into several when trying
to build binary distributions for cross compilers (compiler host != compiler target).

This is also where the relocatable topic came up. If the build system would build ghc
in such a way that I could just take the stage1/stage2/stageN folder and relocate it
into it's final place.  Building binary distributions would be a simple as packaging
up that folder.  If we want to do some additional configuration on the install host,
we could add an automake script, but it wouldn't be strictly necessary.  For the case
of cross compilers, I'm pretty much sure, I do want almost none of the current 
distrib configure script. My toolchain is locked down pretty hard, so I know the tools
I *need* to use.  Anything that the configure script at install time could mess up
is just going to result in more issues down the line.

Now as mentioned I was able to get most of the relocatable logic sorted by dropping
the wrapper script, and have ghc determine it's topdir on it's own (for mac and linux)
by reusing the codepaths already in place for windows.  The package db *does* understand
$topdir and ${pkgroot}.  Again reusing the $topdir logic, present for windows, allows
relocatable package databases.

Now the final issue that came up is dynamic libraries, which on macOS to some degree
encode their location.  However at build time the layout of libraries is sufficiently
different from the layout of libraries once installed.  Which would require to do
some additional path manipulation with the insall_name_tool at install time.

To make this easier, it would be preferable to have the same layout at build time as
it will be at install time.

Another issue I have with the current build system is, that everything is built *in
tree*.  This means that a) if cleaning doesn't work properly, I might have some
stale data lurking around and b) I am unable to build multiple configurations from
the same source tree (or modifications thereof) without resorting to some commit/push/pull
on local copies of the source tree, or wiping out the source tree for each build.

Therefore, after trying to relocate build artifacts in the current build system such
that build and install layouts are more similar.  I hereby declare defeat.  Not only
would this change the way the current build system works quite a bit, it's als pretty
hard to refactor the current aged build system in that way, and I believe it would
result in a number of additional bugs.  And finally, even if that change would make
it into GHC, Hadrian would have to be adapted to it as well.

I have now started trying to graft this different layout ontop of Hadrian.  If this
works out as I hope.  I believe it would drastically simplify the installation rules
as well as binary distribution rules in Hadrian.  It might also provide me with some
knowledge about Hadrian, and how much I like/dislike it over the current make system.

Maybe this is the direction we need to take to make Hadrian a viable option for the
build system.  Otherwise I fear Hadrian will never make it into ghc, if the current
build system keeps evolving and Hadrian fails to keep up.  Ideally we'd probably have
features in Hadrian that the current build system is lacking, which make Hadrian the
attractive alternative. E.g. moving Hadrian to "can do X better" instead of "can also
build GHC, but doesn't have all the features that the current build system has".

Cheers,
 Moritz

> On Oct 25, 2017, at 8:12 AM, Manuel M T Chakravarty <chak at justtesting.org> wrote:
> 
> That doesn’t mean it can’t be used for cross-compilation once it is in the tree.
> 
>> Am 25.10.2017 um 11:06 schrieb Brandon Allbery <allbery.b at gmail.com>:
>> 
>> Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires. 
>> 
>> On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty <chak at justtesting.org> wrote:
>> Why are you saying that? 
>> 
>> I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.
>> 
>> Manuel
>> 
>>> Brandon Allbery <allbery.b at gmail.com>:
>>> 
>>> On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <moritz.angermann at gmail.com> wrote:
>>> However, I am now again at the point where I start hacking on the build
>>> system, while Hadrian is imminent.  And this is quite depressing
>>> 
>>> Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.
>>> 
>>> -- 
>>> brandon s allbery kf8nh                               sine nomine associates
>>> allbery.b at gmail.com                                  ballbery at sinenomine.net
>>> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> 
>> 
>> 
>> 
>> -- 
>> brandon s allbery kf8nh                               sine nomine associates
>> allbery.b at gmail.com                                  ballbery at sinenomine.net
>> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs