[Haskell-cafe] How can I improve the pipes's performance with a huge file?
zhangjun.julian at gmail.com
Fri Nov 14 13:14:17 UTC 2014
I change Map to Strict，it be little fast when test with 18M rows, but it hanged again with 40M rows.
Do you have any other advice?
> 在 2014年11月14日，下午5:52，Tom Ellis <tom-lists-haskell-cafe-2013 at jaguarpaw.co.uk> 写道：
> On Fri, Nov 14, 2014 at 05:43:15PM +0800, zhangjun.julian wrote:
>> But when I test with whole file (40M rows), it take more than 20 minutes and not finished yet.
>> It take more than 9G memorys, and the disk is also busy all time.
>> import qualified Data.Map as DM
> At the very least you should be using
> import qualified Data.Map.Strict as DM
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
More information about the Haskell-Cafe