[Haskell-cafe] How can I improve the pipes's performance with a huge file?
tom-lists-haskell-cafe-2013 at jaguarpaw.co.uk
Fri Nov 14 13:51:20 UTC 2014
On Fri, Nov 14, 2014 at 09:14:17PM +0800, zhangjun.julian wrote:
> Dear Tom
> I change Map to Strict，it be little fast when test with 18M rows, but it hanged again with 40M rows.
> Do you have any other advice?
Dear Zhangjun Julian,
Perhaps too much of the output string is being kept around when it is
printed. I would try
mapM_ (\((x,y), i) -> hPutStrLn writeHadle (show x ++ "," ++ show y ++ "," ++ show i))
hPutStr writeHadle $ mapToString rt
Apart from that, I don't have any other ideas. Can you determine whether
the large memory usage comes from the Pipe or from the printing of the
More information about the Haskell-Cafe