[Haskell-cafe] out of core computing in haskell
droundy at darcs.net
Mon Aug 13 17:48:10 EDT 2007
In the second case, if your numerical data is just a big array, why not
just mmap it? Unless you're running on a 32 bit machine, or have a
*seriously* large amount of data, that seems like the easiest option.
Although if you're talking about mutable data, that's a whole 'nother can
of worms... but that's true even if your data fits into memory.
You could look at Data.ByteString to get some idea as to how to wrap a pure
interface around a disk-backed chunk of read-only memory. Or just read
about the ffi, it's not too hard to figure out.
On Mon, Aug 13, 2007 at 01:27:03PM -0700, Carter T Schonwald wrote:
> The main two use cases I have in mind are
> 1) really really really big abstract syntax trees or proof trees (a la
> compilers or theorem provers)
> 2) random access to numerical data
> does that help clarify what i'm asking about? In each case, is there a
> standard way of dealing with this?
> in the case of (1) the sensible way seems to me to do some sort of zipper
> representation which loads adjacent nodes to some depth surrounding your
> current location in the tree, and in the case of (2) it seems like the
> sensible way would be load subsequences of the data into memory.
More information about the Haskell-Cafe