[Haskell-cafe] Very large data structures in the heap
Dr. Olaf Klinke
o.klinke at dkfz-heidelberg.de
Tue Feb 17 10:05:45 UTC 2015
I have a function that builds a (cyclic) data structure for searching
text. The resulting object can be serialized and re-built. Real-world
problems would, however, result in a tree with about 10^9 nodes. I take it
that standard data type declarations are not very memory-efficient, so my
(fully evaluated) search tree would not fit into memory as a heap object.
Its serialised form should be of tractable size, e.g. by storing an array
of labels and the tree structure as a bit array of parentheses.
I'd like to separate serialization (IO) from traversal of the stucture
(pure), but how can one avoid the entire heap object from being built?
Can lazyness be exploited?
More information about the Haskell-Cafe