[Haskell-cafe] Very large data structures in the heap

Dr. Olaf Klinke o.klinke at dkfz-heidelberg.de
Tue Feb 17 10:05:45 UTC 2015


I have a function that builds a (cyclic) data structure for searching 
text. The resulting object can be serialized and re-built. Real-world 
problems would, however, result in a tree with about 10^9 nodes. I take it 
that standard data type declarations are not very memory-efficient, so my 
(fully evaluated) search tree would not fit into memory as a heap object. 
Its serialised form should be of tractable size, e.g. by storing an array 
of labels and the tree structure as a bit array of parentheses.

I'd like to separate serialization (IO) from traversal of the stucture 
(pure), but how can one avoid the entire heap object from being built? 
Can lazyness be exploited?

Olaf


More information about the Haskell-Cafe mailing list