cabal-install: Conduits/IO concurrency
haskell at benmachine.co.uk
Sun Apr 15 10:43:33 CEST 2012
I'm not at all an authority on this, but one thing to keep in mind is
that it's probably a good idea to keep the transitive package
dependency list of cabal-install as small as possible, otherwise it's
a nuisance to install. In particular, cabal-install is in the Haskell
Platform, so any packages it depends on would have to be distributed
with the HP as well. I doubt the conduit package is stable enough yet
to manage that.
You could possibly introduce some degree of concurrency into the use
of HTTP via the stream hooks interface:
On Sun, Apr 15, 2012 at 12:43 AM, Eyal Lotem <eyal.lotem at gmail.com> wrote:
> Currently it seems that cabal-install lacks IO concurrency in its various
> operations (i.e: download packages concurrently to their compilation).
> Specifically, I found it to be a major bottleneck during "cabal update"
> operations. The download, decompression, and use of the decompressed index
> are all serially executed, and all take quite a bit of time, and could
> probably be executed in parallel easily.
> To make "cabal update" execute the download and decompression concurrently,
> it would be nice to use a streaming decompressor on a stream download.
> The approach I had in mind was replacing the use Network.HTTP and zlib with
> use of the conduits counterparts.
> Additionally, use of conduits can probably ease addition of status reports
> (X% completed) by injecting progress reporters in the conduit pipeline.
> "cabal update" progress could be less opaque that way.
> My question is: Would patches that add conduit use and demonstrate
> performance benefits be welcome?
> I don't want to embark on this adventure if there's a good reason to avoid
> it in the first place.
> Libraries mailing list
> Libraries at haskell.org
More information about the Libraries