[commit: ghc] master: rts/posix: Use less aggressive backoff schedule for heap reservation sizing (730781b)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/730781b42a003604cfa047a02280757a07b09581/ghc

>---------------------------------------------------------------

commit 730781b42a003604cfa047a02280757a07b09581
Author: Ben Gamari <bgamari.foss at gmail.com>
Date:   Wed May 30 12:30:52 2018 -0400

    rts/posix: Use less aggressive backoff schedule for heap reservation sizing
    
    When we allocate the heap on POSIX platforms we generally just ask for a
    1TB chunk of address space and call it a day. However, if the user has
    set a ulimit then this request will fail. In this case we would
    previously try successively smaller allocation requests, reducing the
    request size by a factor of two each time.
    
    However, this means that GHC will significantly allocate a significantly
    smaller heap than the available physical memory size in some
    circumstances.  Imagine, for instance, a machine with 512 GB of physical
    memory but a ulimit of 511 GB: we would be limited to a 256 GB heap.
    
    We now use a less aggressive back-off policy, reducing by one-eighth the
    last allocation size each try.
    
    Thanks to luispedro for the suggested approach.
    
    Test Plan: Validate
    
    Reviewers: simonmar, erikd
    
    Subscribers: rwbarton, thomie
    
    GHC Trac Issues: #14492
    
    Differential Revision: https://phabricator.haskell.org/D4215


>---------------------------------------------------------------

730781b42a003604cfa047a02280757a07b09581
 rts/posix/OSMem.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/rts/posix/OSMem.c b/rts/posix/OSMem.c
index 1df18ab..9ae9a4b 100644
--- a/rts/posix/OSMem.c
+++ b/rts/posix/OSMem.c
@@ -514,9 +514,14 @@ void *osReserveHeapMemory(void *startAddressPtr, W_ *len)
         if (at == NULL) {
             // This means that mmap failed which we take to mean that we asked
             // for too much memory. This can happen due to POSIX resource
-            // limits. In this case we reduce our allocation request by a factor
-            // of two and try again.
-            *len /= 2;
+            // limits. In this case we reduce our allocation request by a
+            // fraction of the current size and try again.
+            //
+            // Note that the previously would instead decrease the request size
+            // by a factor of two; however, this meant that significant amounts
+            // of memory will be wasted (e.g. imagine a machine with 512GB of
+            // physical memory but a 511GB ulimit). See #14492.
+            *len -= *len / 8;
         } else if ((W_)at >= minimumAddress) {
             // Success! We were given a block of memory starting above the 8 GB
             // mark, which is what we were looking for.