[Haskell-cafe] out of core computing in haskell

[David Roundy]

In the second case, if your numerical data is just a big array, why not
just mmap it? Unless you're running on a 32 bit machine, or have a
*seriously* large amount of data, that seems like the easiest option.
Although if you're talking about mutable data, that's a whole 'nother can
of worms... but that's true even if your data fits into memory.

You could look at Data.ByteString to get some idea as to how to wrap a pure
interface around a disk-backed chunk of read-only memory.  Or just read
about the ffi, it's not too hard to figure out.

David

On Mon, Aug 13, 2007 at 01:27:03PM -0700, Carter T Schonwald wrote:
> The main two use cases I have in mind are 
> 1) really really really big abstract syntax trees or proof trees (a la
> compilers or theorem provers)
> 2) random access to numerical data 
> 
> does that help clarify what i'm asking about? In each case, is there a
> standard way of dealing with this?
> in the case of (1) the sensible way seems to me to do some sort of zipper
> representation which loads adjacent nodes to some depth surrounding your
> current location in the tree, and in the case of (2) it seems like the
> sensible way would be load subsequences of the data into memory.