[Haskell-cafe] Data.List / Map: simple serialization?

[Max Bolingbroke]

Hi Dmitri,

On 9 June 2011 09:13, Dmitri O.Kondratiev <dokondr at gmail.com> wrote:
> I wonder how Haskell will distribute memory between the buffer for
> sequential element access (list elements, map tree nodes) and memory for
> computation while reading in list, Data.Map from file?

Your list only has 30,000 elements. From the description of the
problem, you traverse the list several times, so GHC will create an
in-memory link list that persists for the duration of all the
traversals. This is OK, because the number of elements in the list is
small.

For the construction of the Map, it sounds like in the worst case you
will have 30,000*30,000 = 900,000,000 elements in the Map, which you
may not want to keep in memory. Assuming "show", "read" and list
creation are lazy enough, and as long as you use the Map linearly GHC
should be able to GC parts of it to keep the working set small. You
should experiment and see what happens.

My advice is just write the program the simple way (with show and
read, and not worrying about memory) and see what happens. If it turns
out that it uses too much memory you can come back to the list with
your problematic program and ask for advice.

Max