[Haskell-cafe] Why does this program eat RAM?

[Jeremy Shaw]

At Tue, 5 Sep 2006 03:03:51 +0000 (UTC),
John Goerzen wrote:
> 
> I have the below program, and I'm trying to run it on an input of about
> 90MB.  It eats RAM like crazy, and I can't figure out why.

I have not looked in detail at your code -- but it could simply be the
fact that String requires gobs of memory to store a string. If you
forced all 90MB into memory at once, I would expect it to take almost
of gig of RAM. (Around a 10-11 fold increase in size).

I suspect this line could be forcing the whole thing into memory:

> wordfreq = map (\x -> (head x, length x)) . group . sort

because sort can not return the first element until it has looked at
all the elements in the list to determine which one should be first.

If you fold a Data.Map or associative list over the word-list, then
you could probably get the lazy behaviour you expect.

j.