Plan of Attack for Parallel Builds
anirishduck at gmail.com
Tue Mar 29 21:54:08 CEST 2011
I've been looking around and trying to develop a plan for parallelizing builds
in cabal-install. Here's my idea so far:
- Parallelize executeInstallPlan. When given a target load average as a flag it
will determine whether it should spawn a worker (if below the target load
average) or wait. If waiting, it will listen to all worker status channels
and print out their current build status and the load average. Once a worker
exits, it will again check the load average and spawn a new thread if
- Rewrite install.*Package and their callees to use the CHP (Communicating
Haskell Process) monad where possible. Use channels to communicate build
status back to the main thread.
- It might be necessary to parse the output of external builds in some way so
that meaningful status can be communicated back to the user.
- Add a default parallel build log path template. Allow the user to specify one
on the command line to override the default. All output of parallel package
builds will be logged in the background silently instead of displayed to the
- On single-threaded (sequential) builds, revert to the old output style. On
multi-threaded builds, display the current status of all running builds, load
averages and nothing else. Possible output:
Building derive-126.96.36.199... [17 of 58]
Building regex-base-0.93.1... [1 of 4]
Building dyre-0.8.6... [5 of 7]
Configuring xdg-basedir-0.2... [in progress]
Dependencies Built: [0 of 9]
Load Average: [3.4/4.0]
Running 4 Jobs.
A possible error message might look like:
derive-188.8.131.52 failed during the building phase.
Log stored in /home/frank/cabal/logs/build/derive-184.108.40.206.log
What does everyone think?
More information about the cabal-devel