[Haskell-cafe] Program using 500MB RAM to process 5MB file

[Sterling Clover]

I also suspect that manyTill is a really bad choice, since it doesn't  
give you anything until the end token. It would be much better if you  
could rewrite your parser in terms of many and many1.

--Sterl

On Apr 2, 2009, at 10:08 PM, Luke Palmer wrote:

> 2009/4/2 <lucas at die.net.au>
> I'm relatively new to haskell so as one does, I am rewriting an
> existing program in haskell to help learn the language.
>
> However, it eats up all my RAM whenever I run the program.
>
> http://hpaste.org/fastcgi/hpaste.fcgi/view?id=3175#a3175
>
> Obviously I'm doing something wrong, but without my magical FP pants I
> don't know what that might be.
>
> (1) You are using plain Strings.  Those things are like 8 bytes per  
> character (or something, someone more knowledgeable can give a more  
> accurate figure).  Use bytestrings (with bytestring-utf8 if you  
> need it) instead.
>
> (2) You are parsing strictly, meaning you have to read the whole  
> input file before anything can be output.  This may be necessary  
> for your application, but Haskell is very strong with streaming  
> applications.  Change to a lazy parser and you will run in constant  
> memory.
>
> (I don't know offhand of any lazy parsing libraries, but I've heard  
> them discussed before, so they're somewhere)
>
> Luke
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe