[Haskell-beginners] Memory tuning

[Daniel Fischer]

On Monday 11 October 2010 16:14:11, Thorsten Hater wrote:
> Not only is the actual memory consumption quite high, but the
> GC/computation ratio is almost 1:1.

You construct a largish map (about 350000 entries), doing a lot (more than 
a million) of insertWith's.
That means frequent rebalancing (for new circumferences) and many updates 
of already present values (which in all likelihood are heap allocated 
Ints), both give the garbage collector a lot of work. Further, you 
construct a lot of lists, which need to be GCed too.
Also, the memory overhead of Maps is nontrivial, I think five or six words 
per item, so a Map Int Int needs about size*8(or 7)*sizeof(Int) bytes.

You can reduce the GC time by specifying a larger allocation area (e.g.
+RTS -A16M -RTS, the default size is 512K) thus making GC less frequent.

> I assume that there is something to be done regarding strictness,

Nothing obvious.
But for the computation pattern you use, since the ratio of possible values 
to occurring values is not too high, using accumArray (on an Unboxed array) 
is far better.

> but a series of experiments with
> BangPatterns return no yields at all (using foldr instead of foldl'
> produces stack overflow)

Yes, using foldr with insert(With)s into Maps is a Bad Idea™