[Haskell-cafe] performance of map reduce

Bulat Ziganshin bulat.ziganshin at gmail.com
Fri Sep 19 15:17:14 EDT 2008


Hello Don,

Friday, September 19, 2008, 9:12:43 PM, you wrote:

>> It is possible to implement a map reduce version that can handle gzipped
>> log files?

> Using the zlib binding on hackage.haskell.org, you can stream multiple
> zlib decompression threads with lazy bytestrings, and combine the
> results.

for gzip decompression you need to decompress all the previous data,
so decompression of single file cannot be multithreaded


-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com



More information about the Haskell-Cafe mailing list