[commit: ghc] master: Add a RTS option -xp to load PIC object anywhere in address space (104599f)

Tue Oct 16 00:00:27 UTC 2018

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/104599f3f157613589e78627c915e4dc20ee54b4/ghc

>---------------------------------------------------------------

commit 104599f3f157613589e78627c915e4dc20ee54b4
Author: Zejun Wu <watashi at fb.com>
Date:   Mon Oct 15 13:52:36 2018 -0400

    Add a RTS option -xp to load PIC object anywhere in address space

    Add a RTS option -xp to load PIC object anywhere in address space. We do
    this by relaxing the requirement of <0x80000000 result of
    `mmapForLinker` and implying USE_CONTIGUOUS_MMAP.

    We also need to change calls to `ocInit` and `ocGetNames` to avoid
    dangling pointers when the address of `oc->image` is changed by
    `ocAllocateSymbolExtra`.

    Test Plan:
    ```
    $ uname -a
    Linux localhost 4.18.8-arch1-1-ARCH #1 SMP PREEMPT Sat Sep 15 20:34:48
    UTC 2018 x86_64 GNU/Linux
    $ cat mk/build.mk
    DYNAMIC_GHC_PROGRAMS = NO
    DYNAMIC_BY_DEFAULT = NO

    GhcRTSWays += thr_debug
    EXTRA_HC_OPTS += -debug
    WAY_p_HC_OPTS += -fPIC -fexternal-dynamic-refs
    $ inplace/bin/ghc-stage2 --interactive -prof +RTS -xp
    GHCi, version 8.7.20180928: http://www.haskell.org/ghc/  :? for help
    ghc-stage2: R_X86_64_32 relocation out of range:
    ghczmprim_GHCziTypes_ZMZN_closure = 7f690bffab59
    Recompile
    /data/users/watashi/ghc/libraries/ghc-prim/dist-install/build/HSghc-prim
    -0.5.3.o with -fPIC -fexternal-dynamic-refs.
    ghc-stage2: unable to load package `ghc-prim-0.5.3'
    $ strace -f -e open,mmap inplace/bin/ghc-stage2 --interactive -prof
    -fexternal-interpreter  -opti+RTS -opti-xp
    ...
    [pid 1355283]
    open("/data/users/watashi/ghc/libraries/base/dist-install/build/libHSbas
    e-4.12.0.0_p.a", O_RDONLY) = 14
    [pid 1355283] mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC,
    MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a84842000
    [pid 1355283]
    open("/data/users/watashi/ghc/libraries/base/dist-install/build/libHSbas
    e-4.12.0.0_p.a", O_RDONLY) = 14
    [pid 1355283] mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC,
    MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a84676000
    ...
    Prelude> System.Posix.Process.getProcessID
    ...
    [pid 1355283]
    open("/data/users/watashi/ghc/libraries/unix/dist-install/build/libHSuni
    x-2.7.2.2_p.a", O_RDONLY) = 14
    [pid 1355283] mmap(NULL, 45056, PROT_READ|PROT_WRITE|PROT_EXEC,
    MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a67d60000
    [pid 1355283]
    open("/data/users/watashi/ghc/libraries/unix/dist-install/build/libHSuni
    x-2.7.2.2_p.a", O_RDONLY) = 14
    [pid 1355283] mmap(NULL, 57344, PROT_READ|PROT_WRITE|PROT_EXEC,
    MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a67d52000
    ...
    ```

    ```
    $ uname -a
    Darwin watashis-iMac.local 18.0.0 Darwin Kernel Version 18.0.0: Wed Aug
    22 20:13:40 PDT 2018; root:xnu-4903.201.2~1/RELEASE_X86_64 x86_64
    $ mv
    /Users/watashi/gao/ghc/libraries/integer-gmp/dist-install/build/HSintege
    r-gmp-1.0.2.0.o{,._DISABLE_GHC_ISSUE_15105}
    $ inplace/bin/ghc-stage2 --interactive  +RTS -xp
    GHCi, version 8.7.20181003: http://www.haskell.org/ghc/  :? for help
    Prelude> System.Posix.Process.getProcessID
    42791
    Prelude> Data.Set.fromList [1 .. 10]
    fromList [1,2,3,4,5,6,7,8,9,10]
    Prelude>
    Leaving GHCi.
    $ inplace/bin/ghc-stage2 --interactive -prof -fexternal-interpreter
    GHCi, version 8.7.20181003: http://www.haskell.org/ghc/  :? for help
    Prelude> System.Posix.Process.getProcessID
    42806
    Prelude> Data.Set.fromList [1 .. 10]
    fromList [1,2,3,4,5,6,7,8,9,10]
    Prelude>
    Leaving GHCi.
    ```

    Also test with something that used to hit the 2Gb limit and it loads
    and runs without problem.

    Reviewers: simonmar, bgamari, angerman, Phyx, hvr, erikd

    Reviewed By: simonmar

    Subscribers: rwbarton, carter

    Differential Revision: https://phabricator.haskell.org/D5195

>---------------------------------------------------------------

104599f3f157613589e78627c915e4dc20ee54b4
 docs/users_guide/8.8.1-notes.rst     |  4 ++
 docs/users_guide/runtime_control.rst | 28 ++++++++++++-
 includes/rts/Flags.h                 | 12 ++++++
 libraries/base/GHC/RTS/Flags.hsc     |  3 ++
 rts/Linker.c                         | 80 ++++++++++++++++++------------------
 rts/RtsFlags.c                       | 11 +++++
 rts/linker/Elf.c                     | 25 ++---------
 rts/linker/MachO.c                   | 21 ++++++----
 rts/linker/SymbolExtras.c            | 61 +++++++++++++++++----------
 9 files changed, 151 insertions(+), 94 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 104599f3f157613589e78627c915e4dc20ee54b4