[commit: ghc] master: rts: Don't madvise if mmap failed (34464fe)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/34464fed463c7be07d4664e2f4b96eaf1acfc37b/ghc

>---------------------------------------------------------------

commit 34464fed463c7be07d4664e2f4b96eaf1acfc37b
Author: Ben Gamari <bgamari.foss at gmail.com>
Date:   Sun May 27 11:48:53 2018 -0400

    rts: Don't madvise if mmap failed
    
    On 32-bit Linux `outofmem` did not fail with the expected out-of-memory
    error message, instead failing with,
    
        outofmem: internal error: getMBlock: mmap: Invalid argument
    
    This happened because, `my_mmap` would attempt to `madvise` even if the
    `mmap` call failed. So while `mmap` returns `ENOMEM` we nevertheless try
    to `madvise`, which clobbers `errno`, giving us the unexpected `EINVAL`
    error. Consequently we don't detect this to be an out-of-memory error.
    
    This should fix #15060.
    
    Test Plan: `make test TEST=outofmem` on i386
    
    Reviewers: simonmar, erikd
    
    Reviewed By: simonmar
    
    Subscribers: rwbarton, thomie, carter
    
    GHC Trac Issues: #15060
    
    Differential Revision: https://phabricator.haskell.org/D4704


>---------------------------------------------------------------

34464fed463c7be07d4664e2f4b96eaf1acfc37b
 rts/posix/OSMem.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/rts/posix/OSMem.c b/rts/posix/OSMem.c
index 9ecd53e..1df18ab 100644
--- a/rts/posix/OSMem.c
+++ b/rts/posix/OSMem.c
@@ -192,16 +192,18 @@ my_mmap (void *addr, W_ size, int operation)
         }
     }
 
-    if (operation & MEM_COMMIT) {
-        madvise(ret, size, MADV_WILLNEED);
+    if (ret != (void *)-1) {
+        if (operation & MEM_COMMIT) {
+            madvise(ret, size, MADV_WILLNEED);
 #if defined(MADV_DODUMP)
-        madvise(ret, size, MADV_DODUMP);
+            madvise(ret, size, MADV_DODUMP);
 #endif
-    } else {
-        madvise(ret, size, MADV_DONTNEED);
+        } else {
+            madvise(ret, size, MADV_DONTNEED);
 #if defined(MADV_DONTDUMP)
-        madvise(ret, size, MADV_DONTDUMP);
+            madvise(ret, size, MADV_DONTDUMP);
 #endif
+        }
     }
 
 #else