[Haskell-cafe] How can I improve the pipes's performance with a huge file?

[Wojtek Narczyński]

On 14.11.2014 23:50, zhangjun.julian wrote:
> If I change my Map to Strict and  call rCount directly( don’t use 
> mapM_ ) the memory will not swell.
>
I used the word "swell" to describe the phenomenon that in Haskell 
pointers can consume vast amounts of memory, especially on 64 bit 
architectures. For example type 
(Bool,Bool,Bool,Bool,Bool,Bool,Bool,Bool), one byte of information, will 
take 136 bytes, unless you fight the laziness feature with ! and UNPACK. 
Okay, this is an evil contrived example, but you get the idea. It is not 
generally accepted nomenclature.

You were hit by what is referred to as "space leak", that is build-up of 
unevaluated closures.

Anyway, I'm glad to hear that your problem is gone.

-- 
Kind reagards,
Wojtek


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20141115/b73f8da1/attachment.html>