[Haskell-cafe] How can I improve the pipes's performance with a huge file?

zhangjun.julian zhangjun.julian at gmail.com
Fri Nov 14 13:14:17 UTC 2014


Dear Tom

I change Map to Strict,it be little fast when test with 18M rows, but it hanged again with 40M rows.

Do you have any other advice?




> 在 2014年11月14日,下午5:52,Tom Ellis <tom-lists-haskell-cafe-2013 at jaguarpaw.co.uk> 写道:
> 
> On Fri, Nov 14, 2014 at 05:43:15PM +0800, zhangjun.julian wrote:
>> But when I test with whole file (40M rows), it take more than 20 minutes and  not finished yet.
>> It take more than 9G  memorys, and the disk is also busy all time.
> 
>> import qualified Data.Map as DM
> 
> At the very least you should be using
> 
>    import qualified Data.Map.Strict as DM
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe



More information about the Haskell-Cafe mailing list