[GHC] #10093: Library configure scripts should not clobber CPPFLAGS nor LDFLAGS

[GHC]

#10093: Library configure scripts should not clobber CPPFLAGS nor LDFLAGS
-------------------------------------+-------------------------------------
              Reporter:  PHO         |             Owner:
                  Type:  bug         |            Status:  new
              Priority:  normal      |         Milestone:
             Component:  Build       |           Version:  7.11
  System                             |  Operating System:  Unknown/Multiple
              Keywords:              |   Type of failure:  Runtime crash
          Architecture:              |        Blocked By:
  Unknown/Multiple                   |   Related Tickets:
             Test Case:              |
              Blocking:              |
Differential Revisions:              |
-------------------------------------+-------------------------------------
 `libraries/base/configure` currently clobbers `CPPFLAGS` and `LDFLAGS`
 when `--with-iconv-includes` and `--with-iconv-libraries` are given, but
 really it shouldn't.

 I will soon submit a patch for this.

 Here's what I stumbled upon (TL;DR):

 This is somewhat related to #2933. I have libiconv installed in
 `/usr/pkg`, which is a non-standard path that `ld.so` normally doesn't
 search libraries for. So I built GHC with
 `CONF_GCC_LINKER_OPTS_STAGE{0,1,2}` set to `-Wl,-rpath,/usr/pkg/lib` so
 that GHC links executables with `RPATH` pointing to `/usr/pkg/lib`.

 The build went fine until it started using `ghc-stage2` and `haddock`:
 they were causing segfaults in `evacuate()` with a probability of about
 70%, which suggested a heap corruption occuring somewhere. I tried
 rebuilding RTS with `-O0 -g`, changed various parameters for GC, etc.
 Nothing changed.

 Then I suddenly realized that `libraries/base/configure` was the root
 cause of the problem. Those `configure` scripts are executed with
 `LDFLAGS` being set to the value of `CONF_GCC_LINKER_OPTS_STAGE2`, which
 is `-Wl,-rpath,/usr/pkg/lib` in my case, but `libraries/base/configure`
 overwrites it to `-L/usr/pkg/lib` when `--with-iconv-
 libraries=/usr/pkg/lib` is given. On the other hand,
 `FP_SEARCH_LIBS_PROTO(iconv)` appends `-liconv` to `$LIBS` so it will be
 linked to any conftest executables that follow, including one which will
 be generated by `AC_CHECK_SIZEOF()`.

 The problem is that if libraries listed in `$LIBS` are in a non-standard
 path while rpath flags are missing (due to `LDFLAGS` being clobbered in
 this case), conftest executables cannot run even if they can be linked.
 And if anything goes wrong during `AC_CHECK_SIZEOF(T)`, it considers
 `sizeof(T)` as 0 unless `T` is known to be an existing type:
 {{{
   ...
   checking for library containing iconv... -liconv
   checking for library containing locale_charset... none required
   checking size of struct MD5Context... 0
   ...
 }}}
 This means `SIZEOF_STRUCT_MD5CONTEXT` gets defined to 0,
 `GHC.Fingerprint.fingerprintData` allocates 0 bytes on the heap,
 `MD5Init`/`Update`/`Final` corrupts the heap and then Bad Things will
 happen.

 I have found the problem on NetBSD/amd64 but I'm sure the same thing
 happens on every ELF platform.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10093>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler