Unreg/registerised hc booting success

Donald Bruce Stewart dons@cse.unsw.edu.au
Mon, 25 Aug 2003 19:00:57 +1000

Hey all,

Here is a (long) summary of how I have bootstraped GHC on
various os/arch combinations using 6.0.1 .hc source, including
getting an unregisterised build, over the weekend.

              1. Generating registerised .hc files

...on a machine with GHC 6.0.1 installed. This process has worked
on an x86 OpenBSD box, and an x86 Linux box.

I used the following build.mk, extracted from the nightly build

    GhcStage1HcOpts = -H48m -fvia-C
    GhcStage2HcOpts = -H48m -fvia-C
    GhcStage3HcOpts = -H48m -fvia-C -keep-hc-file
    GhcLibHcOpts    = -H48m -fvia-C -keep-hc-file
    GhcLibWays = 


    # add this after the normal build, and after 'gmake clean' in
    # utils, in order to get the proper symbols in utils
    #SRC_HC_OPTS+=-keep-hc-file -fvia-C -H48m

    BIN_DIST = 1

The process was as follows (presuming you have untarred a nice,
shiny ghc-6.0.1 src):

    $ ./configure
    $ make bootstrap3

This was all good, but the problem was: no .hc files were made in
ghc/utils. Setting SRC_HC_OPTS the whole time was no good, as we
ended up with wrong symbols in ghc/utils/*.hc. The solution is to

    $ ./configure --with-hc=`pwd`/ghc/compiler/ghc-inplace
      # then uncomment out the SRC_HC_OPTS line of mk/build.mk
    $ ( cd ghc/utils ; make clean ; make )
    $ make hc-file-bundle Project=Ghc

You should now find a proper .hc file bundle in the fptools
directory, ready to install over the full Haskell source.

Also, I had a weird problem with parser-generated files missing
in the .hc build. This was due to the hc-file-bundle from the
shadow tree containing *links* to the now-missing parser
generated .hs files, not the files themselves. The solution is to
do the above .hc generating build inside a real tree, not a
shadow (lndir'd) tree.

          2. Bootstrapping with registerised .hc files

I had to modify 3 files:
  distrib/hc-build: to correctly touch stage1 files for "make install" 
  mk/bootstrap.mk: was out of date for 6.0.1 libraries
  libraries/readline/Makefile: didn't like stub files on booting

      # untar the .hc source of a clean Haskell source tree 
    $ distrib/hc-build --prefix=`pwd`
    $ make install stage=1

With these steps, and the following 3 patches, I was able to
boostrap from 6.0.1 .hc files successfully on both Linux and
OpenBSD x86 boxes. 

                   3. Unregisterised building

Now by following the above instructions, and applying the
appended patches, along with setting "GhcUnregisterised = YES" in
the first build.mk, and then again in the bootstrapping
distrib/hc-build, *and* running distrib/hc-build with
--enable-hc-boot-unregisterised I was able to build an
unregisterised compiler and libraries successfully on an x86 box
(didn't try on Linux this time). I am about to see if I can take
these unreg files and boot a sparc straight to 6.0.1.

The main problem in all of this was generating correct .hc files
in the first place :)


The patches (to get .hc building working) for SimonM to look at:


--- distrib/hc-build.orig	2003-07-25 20:23:30.000000000 +1000
+++ distrib/hc-build	2003-08-25 13:47:44.000000000 +1000
@@ -72,11 +72,6 @@
 	      primop-usage.hs-incl  \
-# The reconfigure step updates a few files, which can lead to
-# unnecessary recompilations.  Touch a bunch of things here to avoid
-# having to recompile stuff that we've already built.
-(cd ghc/compiler; touch $PRIMOP_BITS parser/hschooks.o prelude/PrimOp.o main/Config.hs main/Config.o ghc-*)
 # Remove the old libraries.  Don't use make clean, because we don't
 # want to delete the .hs files generated from the .hsc files, because
 # we don't have hsc2hs built yet.
@@ -95,7 +90,10 @@
 # Now we can build hslibs (hsc2hs is required, so must be after ghc/utils)
 $MAKE -C hslibs  boot all
+# The reconfigure step updates a few files, which can lead to
+# unnecessary recompilations.  Touch a bunch of things here to avoid
+# having to recompile stuff that we've already built.
 # Avoid relinking the compiler during 'make install':
-(cd ghc/compiler; touch parser/hschooks.o ghc-*)
+(cd ghc/compiler/; touch $PRIMOP_BITS main/Config.hs stage1/parser/hschooks.o stage1/prelude/PrimOp.o stage1/main/Config.o stage1/ghc-* ghc-inplace)
 # At this point, the tree should be safe to do 'make install' in.
--- libraries/readline/Makefile.orig	2003-08-25 13:47:07.000000000 +1000
+++ libraries/readline/Makefile	2003-08-25 13:47:14.000000000 +1000
@@ -27,7 +27,9 @@
     SRC_CC_OPTS += -I$(ReadlineIncludePath)
     SRC_HSC2HS_OPTS += -I$(ReadlineIncludePath)
+ifneq "$(BootingFromHc)" "YES"
   STUBOBJS += System/Console/Readline_stub.$(way_)o
   CLEAN_FILES += System/Console/Readline_stub.[ch] $(STUBOBJS)
   EXCLUDED_SRCS += System/Console/Readline.hsc System/Console/Readline_stub.c
--- mk/bootstrap.mk.orig	2003-07-25 00:08:25.000000000 +1000
+++ mk/bootstrap.mk	2003-08-25 13:46:22.000000000 +1000
@@ -17,6 +17,8 @@
 # Reset TOP
+include $(TOP)/mk/suffix.mk
 # -----------------------------------------------------------------------------
 # Set the platform-specific options to send to the C compiler.  These should
 # match the list in machdepCCOpts in ghc/compiler/DriverFlags.hs.
@@ -68,7 +70,8 @@
-SRC_CC_OPTS += -I$(FPTOOLS_TOP_ABS)/ghc/includes -I$(FPTOOLS_TOP_ABS)/libraries/base/include -I$(FPTOOLS_TOP_ABS)/libraries/unix/include
+SRC_CC_OPTS += -I$(FPTOOLS_TOP_ABS)/ghc/includes -I$(FPTOOLS_TOP_ABS)/libraries/base/include -I$(FPTOOLS_TOP_ABS)/libraries/unix/include -I$(FPTOOLS_TOP_ABS)/libraries/readline/include
 # -----------------------------------------------------------------------------
 # Linking: we have to give all the libraries explicitly.
@@ -89,6 +92,10 @@
    -L$(FPTOOLS_TOP_ABS)/libraries/base		\
    -L$(FPTOOLS_TOP_ABS)/libraries/base/cbits	\
    -L$(FPTOOLS_TOP_ABS)/libraries/haskell98	\
+   -L$(FPTOOLS_TOP_ABS)/libraries/haskell-src	\
+   -L$(FPTOOLS_TOP_ABS)/libraries/readline	\
+   -L$(FPTOOLS_TOP_ABS)/libraries/unix         \
+   -L$(FPTOOLS_TOP_ABS)/libraries/unix/cbits   \
    -u "$(UNDERSCORE)GHCziBase_Izh_static_info" \
    -u "$(UNDERSCORE)GHCziBase_Czh_static_info" \
    -u "$(UNDERSCORE)GHCziFloat_Fzh_static_info" \
@@ -121,7 +128,7 @@
    -u "$(UNDERSCORE)GHCziWeak_runFinalizzerBatch_closure" \
    -u "$(UNDERSCORE)__stginit_Prelude"
-HC_BOOT_LIBS = -lHShaskell98 -lHSbase -lHSbase_cbits -lHSrts -lgmp -lm $(EXTRA_HC_BOOT_LIBS)
+HC_BOOT_LIBS = -lHShaskell98 -lHSreadline -lHShaskell-src -lHSunix -lHSunix_cbits -lHSbase -lHSbase_cbits -lHSrts -lgmp -lm -lreadline -lncurses $(EXTRA_HC_BOOT_LIBS)
 ifeq "$(GhcLibsWithReadline)" "YES"
 HC_BOOT_LIBS += $(patsubst %, -l%, $(LibsReadline))