Building GHC in a small amount of memory, in parts.

[Aaron Friel]

I am trying to build GHC-HEAD and produce a reliable Dockerfile (see: docker.io) to allow others to use GHC-HEAD and also to make my own build process more reliable. I would like advice/assistance on two things to help this. For reference, I am using the stackbrew/ubuntu:13.10 image as my starting point, and bootstrapping GHC to HEAD using the current GHC (7.6.x) in Ubuntu's repositories.

First issue: I am having issues with constraining the resource consumption during the build and have resorted to a few hacks such as replacing ld with gold (or rather, a script that filters out --hash-size arguments) and a custom build.mk file that uses -fasm for stage1. I would also like the final (stage 2?) result to be compiled with llvm, perhaps even without requiring gcc installed at all. How can I continue to push down the resource requirements of a build, and end up with a GHC that represents the bleeding edge? (That is what I want to be testing against, finding bugs for, etc.)

Here is my build.mk:

################## BUILD.MK

# Default to a very fast build for compilation and stage1:
SRC_HC_OPTS        = -H64m -O0
GhcStage1HcOpts    = -O -fasm

# But then use perf-llvm settings for remaining settings:

GhcStage2HcOpts = -O2 -fllvm
GhcHcOpts       = -Rghc-timing
GhcLibHcOpts    = -O2 -fllvm -XGenerics
GhcLibWays     += p

DYNAMIC_BY_DEFAULT   = NO
DYNAMIC_GHC_PROGRAMS = NO

# Disable building all docs

HADDOCK_DOCS       = NO
BUILD_DOCBOOK_HTML = NO
BUILD_DOCBOOK_PS   = NO
BUILD_DOCBOOK_PDF  = NO

################## BUILD.MK

Second issue: These seem to bring memory usage down, but the monolithic ".\make" command is difficult to debug and very time consuming. When building a container image from a Dockerfile, the commands used are cached so that subsequent attempted builds use the intermediate results. So, even though my Dockerfile starts out with "apt-get update; apt-get upgrade -y", I don't have to do that every time I retry my build.

Currently my Dockerfile pulls all the requisite packages from Ubuntu's repos, downloads and builds the latest cabal-install and builds alex, happy, hscolour and haddock, gets the ghc repo from GitHub, does "perl boot" ".\configure" and then I hit a wall with the monstrous amount of work that ".\make" does. I have tried splitting it up by doing ".\make stage=1", ".\make 1", and various combinations, but I hit dependency issues. Is there a way to break up the build into multiple commands?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131219/dd3dbad3/attachment.html>