[Haskell-cafe] GHC FreeBSD memory model (was: 8.4.3 release)

[Ben Gamari]

Viktor Dukhovni <ietf-dane at dukhovni.org> writes:

> On Thu, Jul 05, 2018 at 09:32:31AM -0400, Ben Gamari wrote:
>
>> We use a much different memory allocation strategy on FreeBSD which
>> (AFAIK) does not allow one to reserve address space without also
>> committing.
>
> Here's a demo of mapping in a 1TB heap on FreeBSD 11.1, on a machine
> with "just" 64GB of RAM and 160GB swap.  I would expect anon memory
> mappings to behave similarly on older systems.
>
Hmm, it's possible this could be fixed fairly easily in that case. The
original reason for disabling the two-step allocator is #12695, where
GHC failed to build due to MAP_NORESERVE being undefined. I had assumed
that this meant that reservation-only mappings weren't defined however
now that I look again at the mmap(2) manpage it looks like I
am probably wrong. MAP_NORESERVE merely flags to the system that swap
space should not be reserved for the mapping.

It looks like this functionality was simply never implemented in FreeBSD
and the flag eventually removed in [1]. However, your example suggests
that the kernel may be doing the right thing by default. It sounds to me
like we could likely remove the assertion enforcing the availability of
MAP_NORESERVE and enable two-step allocation on FreeBSD.

My only question is how one performs reservation-and-commit mappings on
FreeBSD. This is important since without committing any access to the
mapping may crash the program in the case of OOM. Does FreeBSD simply
not provide a way to safely map known-good memory?

Regardless, thanks for bringing this up!

Cheers,

 -Ben


[1] https://reviews.freebsd.org/D848
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20180705/578dd86c/attachment.sig>