[Haskell-cafe] Processing of large files

Ketil Malde ketil+haskell at ii.uib.no
Mon Nov 1 14:48:44 EST 2004

"Alexander N. Kogan" <alexander at kogan.nnov.ru> writes:

> How should I modify it to make it useful on large file?
> It eats too much memory...

> procFile =
>     putStrLn       .
>     show       .
>     foldl merge []      .

>     words

foldl is infamous for building the complete list, before evaluating
anything. Did you try foldr or foldl' instead?   Also, you may want to
make 'merge' more strict; I suspect you build lazy tuples that look like
("word",1+1+1+1+...) and only get evaluated at the end.

Using heap profiling will probably give you some hints (well
documented in the GHC manual, ask if you get stuck)

If I haven't seen further, it is by standing in the footprints of giants

More information about the Haskell-Cafe mailing list