cabal-install: Conduits/IO concurrency

[Eyal Lotem]

Hey,

Currently it seems that cabal-install lacks IO concurrency in its various
operations (i.e: download packages concurrently to their compilation).

Specifically, I found it to be a major bottleneck during "cabal update"
operations. The download, decompression, and use of the decompressed index
are all serially executed, and all take quite a bit of time, and could
probably be executed in parallel easily.

To make "cabal update" execute the download and decompression concurrently,
it would be nice to use a streaming decompressor on a stream download.

The approach I had in mind was replacing the use Network.HTTP and zlib with
use of the conduits counterparts.

Additionally, use of conduits can probably ease addition of status reports
(X% completed) by injecting progress reporters in the conduit pipeline.
"cabal update" progress could be less opaque that way.


My question is: Would patches that add conduit use and demonstrate
performance benefits be welcome?

I don't want to embark on this adventure if there's a good reason to avoid
it in the first place.

Eyal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/libraries/attachments/20120415/02129182/attachment.htm>