[Haskell-cafe] Re: FFI and heap memory usage limit
dons at galois.com
Mon Jun 29 16:32:40 EDT 2009
> On Fri, 2009-06-26 at 13:00 +0100, Simon Marlow wrote:
> > Maybe bzlib allocates using malloc()? That would not be tracked by
> > GHC's memory management, but could cause OOM.
> Yes it does.
> > Another problem is that if you ask for a large amount of memory in one
> > go, the request is usually honoured immediately, and then we GC shortly
> > afterward. If this is the problem for you, please submit a ticket and
> > I'll see whether it can be changed. You could work around it by calling
> > System.Mem.performGC just before allocating the memory.
> What I'd like is a way to inform the GC about how much memory there is
> attached to a ForeignPtr. The amount would contribute to the GC's
> decision on when to collect. This would solve the bzlib case and also
> the same problem in other bindings like cairo (big pixmaps) and gtk2hs
> (lots and lots of medium-sized foreign objects).
> Obviously it's not a total solution since not all bindings can easily
> discover the size of the foreign allocations, but in the case of zlib,
> bzlib, cairo and gtk2hs it would be possible. It also does not have to
> be 100% accurate.
> For implementation, perhaps keep in the rts an extra foreign memory
> count (which gets used by the GC in its accounting). Then on a foreign
> allocation we add to the count. An extra finaliser on the ForeignPtr
> could then decrement the foreign memory count. Another, more local,
> approach might be to keep the accounting info in a data structure
> directly, eg:
> data ForeignAccounting#
> which would contain a count. This type would be recognised by the GC.
> These could be embedded into data structures anywhere to account for the
> size of foreign allocations, in particular into the ForeignPtrContents.
Sounds like the solution we thought through a few months ago. I'd
support this -- it would be useful.
More information about the Haskell-Cafe