Plan of Attack for Parallel Builds

[Bryan O'Sullivan]

On Tue, Mar 29, 2011 at 12:54 PM, Frank Murphy <anirishduck at gmail.com>wrote:

> I've been looking around and trying to develop a plan for parallelizing
> builds
> in cabal-install.


Something I'd love to see, for sure. But the devil's in the details; see
below.


> - Parallelize executeInstallPlan. When given a target load average as a
> flag it
>  will determine whether it should spawn a worker (if below the target load
>  average) or wait.


Load average is a very very bad method to use, because it's very slow to
respond to changes in real load on Unix, and it doesn't exist at all on
Windows. Do like "make" instead and just accept a "-j" parameter for the
maximum number of jobs to run simultaneously.

- Rewrite install.*Package and their callees to use the CHP (Communicating
>  Haskell Process) monad where possible. Use channels to communicate build
>  status back to the main thread.
>

Why not just use MVar or Chan?


> - It might be necessary to parse the output of external builds in some way
> so
>  that meaningful status can be communicated back to the user.


Yuck, no. Too fragile. Just check the exit status of a process.

- Add a default parallel build log path template. Allow the user to specify
> one
>  on the command line to override the default. All output of parallel
> package
>  builds will be logged in the background silently instead of displayed to
> the
>  user.
> - On single-threaded (sequential) builds, revert to the old output style.
> On
>  multi-threaded builds, display the current status of all running builds,
> load
>  averages and nothing else.


No to both of these, too. It's desirable that build outputs shouldn't
scribble on each other via excessive interleaving, but telling people to go
read a log file because a build failed is going to give them a very bad
experience. For instance, that would defeat the usual way that users of
Emacs and other IDEs jump to the first error.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/cabal-devel/attachments/20110329/c2b855f9/attachment.htm>