[GHC] #3658: Dynamically link GHCi (and use system linker) on platforms that support it

[GHC]

#3658: Dynamically link GHCi (and use system linker) on platforms that support it
-------------------------------------------------+-------------------------
        Reporter:  simonmar                      |            Owner:
            Type:  task                          |           Status:  new
        Priority:  high                          |        Milestone:  7.8.1
       Component:  GHCi                          |          Version:
      Resolution:                                |  6.10.4
Operating System:  Unknown/Multiple              |         Keywords:
 Type of failure:  None/Unknown                  |  dynamic link
       Test Case:                                |     Architecture:
        Blocking:  781, 1883, 2283, 3242, 3333,  |  Unknown/Multiple
  3372, 3654, 4244, 5062, 5197, 5435, 6107,      |       Difficulty:
  7043, 7056, 7072, 7097, 7103, 7134, 7316,      |  Unknown
  7329, 7389, 7475, 7687                         |       Blocked By:  5987
                                                 |  Related Tickets:
-------------------------------------------------+-------------------------

Old description:

> In 6.14.1 we should switch to shipping a dynamically linked GHCi binary,
> on those platforms for which dynamic linking is supported (currently
> Linux; MacOS X and Windows support is in progress).
>
> == Advantages ==
>
>  * The GHCi binary is smaller
>  * some packages don't need to be loaded on startup: lower memory use
>  * GHCi startup might be quicker (or it might not)
>  * some hacks due to having two copies of the base package are not
>    necessary (see `rts/Globals.c`)
>  * We might save some space in the distributions.
>  * It takes us a step closer to not needing the RTS linker at all
>  * It takes us a step closer to using dynamic linking by default, which
>    is where we want to go ultimately
>
> == Potential Issues ==
>
>  * Do we run into any problems with GHCi and the user program sharing the
> same
>    stdin/stdout/stderr handles?  Do we need to virtualise these
> explicitly in the
>    GHCi front end?
>
>  * We cannot revert CAFs in packages that are shared by GHC and the user
> program.
>    There are some old non-working hacks related to reverting CAFs when
> GHCi is
>    dynamically linked (see `KeepCAFsForGHCi`) that
>    need to be cleaned out.  CAFs can only be reverted in code loaded by
> the RTS
>    linker.  We need to think about whether this is a necessary feature or
> not:
>    we have never supported CAF reverting for interpreted code anyway.
> One
>    reason to have it was so that you can recover after saying
> `getContents` at
>    the GHCi prompt, but we can provide other ways to work around that.
>
>  * There will be installation/binary-dist issues to resolve; currently we
> don't
>    install any dynamically-linked binaries.
>
> == Open questions ==
>
>  * Whether we continue to use the same binary for GHC and GHCi is an open
> question:
>    it would be possible to provide a separate statically-linked GHC
> binary if
>    performance of the dynamically-linked version was an issue.
>
>  * We might as well dynamically-link Haddock, and the other tools that
> come
>    with GHC too.

New description:

 In 6.14.1 we should switch to shipping a dynamically linked GHCi binary,
 on those platforms for which dynamic linking is supported (currently
 Linux; MacOS X and Windows support is in progress).

 == Advantages ==

  * The GHCi binary is smaller
  * some packages don't need to be loaded on startup: lower memory use
  * GHCi startup might be quicker (or it might not)
  * some hacks due to having two copies of the base package are not
    necessary (see `rts/Globals.c`)
  * We might save some space in the distributions.
  * It takes us a step closer to not needing the RTS linker at all
  * It takes us a step closer to using dynamic linking by default, which
    is where we want to go ultimately

 == Potential Issues ==

  * Do we run into any problems with GHCi and the user program sharing the
 same
    stdin/stdout/stderr handles?  Do we need to virtualise these explicitly
 in the
    GHCi front end?

  * We cannot revert CAFs in packages that are shared by GHC and the user
 program.
    There are some old non-working hacks related to reverting CAFs when
 GHCi is
    dynamically linked (see `KeepCAFsForGHCi`) that
    need to be cleaned out.  CAFs can only be reverted in code loaded by
 the RTS
    linker.  We need to think about whether this is a necessary feature or
 not:
    we have never supported CAF reverting for interpreted code anyway.  One
    reason to have it was so that you can recover after saying
 `getContents` at
    the GHCi prompt, but we can provide other ways to work around that.
 However, if
    we eliminated HEAP_ALLOCED (#8199), as a side effect of this
 implementation,
    this might be doable, as we no longer have to reach our fingers into
 the
    data section of dynamically linked in libraries to revert a CAF; all of
 the CAFs
    are copied to dynamic memory space first. (The current implementation
 doesn't
    work for that, however!)

  * There will be installation/binary-dist issues to resolve; currently we
 don't
    install any dynamically-linked binaries.

 == Open questions ==

  * Whether we continue to use the same binary for GHC and GHCi is an open
 question:
    it would be possible to provide a separate statically-linked GHC binary
 if
    performance of the dynamically-linked version was an issue.

  * We might as well dynamically-link Haddock, and the other tools that
 come
    with GHC too.

--

Comment (by ezyang):

 If we eliminated HEAP_ALLOCED (#8199), as a side effect of this
 implementation, this might be doable, as we no longer have to reach our
 fingers into the
 data section of dynamically linked in libraries to revert a CAF; all of
 the CAFs
 are copied to dynamic memory space first. (The current implementation
 doesn't
 work for that, however!)

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/3658#comment:41>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler