[Haskell-cafe] mapping large structures into memory

[Don Stewart]

warrensomebody:
> I've dabbled in haskell, but am by no means an expert. I was hoping  
> someone here could help me settle this debate so that we can more  
> seriously consider haskell for a next version of an application we're  
> building....
>
> I would like to understand better what its capabilities are for directly 
> mapping and managing memory. For instance, I would like mmap many large 
> files into memory and mutate their internals directly... without needing 
> to reallocate them (or chunks of them) in the haskell heap, and without 
> resorting to a byte-array and byte-offset representation. Furthermore, I 
> might also like to map intrinsic haskell data structures into this mmap'd 
> memory such that standard library functions can manipulate them (perhaps 
> in a purely functional way, e.g. treating them as haskell arrays of 
> smaller foreign structures).
>
> I understand that the foreign function interface has the ability to  
> marshall/unmarshall C structs, but I'm unsure of the memory implications 
> of using this mechanism. Our application has a very large footprint, and 
> reallocating some or all of these mapped files is a non-starter. Thanks,

It is entirely possible to use mmap to map structures into memory.
Thanks to the foreign function interface, there are well-defined
semantics for calling to and from C.

The key questions would be:

 * what is the type and representation of the data you wish to map    
 * what operations on them

-- Don