[commit: ghc] master: Treat EPERM error from mmap as an OOM (#7500) (acb91b9)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/acb91b920ebac992c52594adf605b2caf98ab4c0/ghc

>---------------------------------------------------------------

commit acb91b920ebac992c52594adf605b2caf98ab4c0
Author: Reid Barton <rwbarton at gmail.com>
Date:   Wed Aug 28 16:16:08 2013 -0400

    Treat EPERM error from mmap as an OOM (#7500)
    
    Linux can give back EPERM from an mmap call when a user program attempts
    to map pages near `mmap_min_addr`, which is a kernel security measure to
    prevent people from mapping pages at address 0. We may do this when we
    hint to mmap what address to map the pages to.
    
    However, it's theoretically possible we're not actually out of memory -
    we could have continuously mapped pages at some other place far away
    from `mmap_min_addr` and succeeded instead. So as an added precaution,
    if mmap for a given addr gives us EPERM, we'll also attempt to map
    *again*, but without the address hint. Maybe the kernel can do the right
    thing.
    
    However, while testing #7500, the amount of free address space we could
    have otherwise used only turns out to be about 139MB. Which isn't really
    a lot. So, given that, we *also* otherwise treat EPERM as an out of
    memory error.
    
    This fixes #7500.
    
    Signed-off-by: Austin Seipp <aseipp at pobox.com>


>---------------------------------------------------------------

acb91b920ebac992c52594adf605b2caf98ab4c0
 rts/posix/OSMem.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/rts/posix/OSMem.c b/rts/posix/OSMem.c
index 26aebc2..5486a15 100644
--- a/rts/posix/OSMem.c
+++ b/rts/posix/OSMem.c
@@ -112,6 +112,27 @@ my_mmap (void *addr, W_ size)
     } else {
 	vm_protect(mach_task_self(),(vm_address_t)ret,size,FALSE,VM_PROT_READ|VM_PROT_WRITE);
     }
+#elif linux_HOST_OS
+    ret = mmap(addr, size, PROT_READ | PROT_WRITE,
+               MAP_ANON | MAP_PRIVATE, -1, 0);
+    if (ret == (void *)-1 && errno == EPERM) {
+        // Linux may return EPERM if it tried to give us
+        // a chunk of address space below mmap_min_addr,
+        // See Trac #7500.
+        if (addr != 0) {
+            // Try again with no hint address.
+            // It's not clear that this can ever actually help,
+            // but since our alternative is to abort, we may as well try.
+            ret = mmap(0, size, PROT_READ | PROT_WRITE,
+                       MAP_ANON | MAP_PRIVATE, -1, 0);
+        }
+        if (ret == (void *)-1 && errno == EPERM) {
+            // Linux is not willing to give us any mapping,
+            // so treat this as an out-of-memory condition
+            // (really out of virtual address space).
+            errno = ENOMEM;
+        }
+    }
 #else
     ret = mmap(addr, size, PROT_READ | PROT_WRITE, 
                MAP_ANON | MAP_PRIVATE, -1, 0);