[Haskell-cafe] Slow IO

[Udo Stenzel]

Daniel Fischer wrote:
> The programme consumed more and more memory (according to top),
> kswapd started to have a higher CPU-percentage than my programme,
> programme died, system yelling 'Speicherzugriffsfehler', top displays 
> 'kswapd<defunct>'.
> I believe that means my programme demanded more memory than I have available 
> (only 256MB RAM + 800MB swap). Is that a segfault or what is the correct 
> term?
> 
> That is probably due to (apart from the stupidity of my IO-code) the large 
> overhead of Haskell lists.

Most certainly not.  I'm pretty sure this is to a bug in your code.
Something retains a data structure which is actually unneeded.  Probably
a case of "foldl" where "foldl'" should be used or a "try" in Parsec
code where it should be left out or a lot of "updateWiths" to a Map,
etc.  Or it could be a bad choice of data structure.  I bet, it's the
map you're using to represent the graph (which you don't even need to
represent at all, btw).


> So the chunk of the file which easily fits into my 
> RAM in ByteString form is too large as a list of ordinary Strings.

The chunk of file should never need to fit into RAM.  If that's a
problem, you also forgot to prime a crucial "foldl".


Udo.
-- 
"Proof by analogy is fraud." -- Bjarne Stroustrup
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20060912/a51892d7/attachment.bin