Parallel building multiple targets

[John Lato]

(FYI, I expect I'm the source of the suggestion that ghc -M is broken)

First, just to clarify, I don't think ghc -M is obviously broken.  Rather,
I think it's broken in subtle, unobvious ways, such that trying to develop
a make-based project with ghc -M will fail at various times in a
non-obvious fashion, at least without substantial additional rules.  For an
example of some of the extra steps necessary to make something like this
work, see e.g. https://github.com/nh2/multishake (which is admittedly for a
more complicated setup, and also has some issues).  The especially
frustrating part is, just when you think you have everything working,
someone wants to add some other tool to a workflow (hsc2hs, .cmm files,
etc), and your build system doesn't support it.

ghc --make doesn't allow building several binaries in one run, however if
you use cabal all the separate runs will use a shared build directory, so
subsequent builds will be able to take advantage of the intermediate output
of the first build.  Of course you could do the same without cabal, but
it's a convenient way to create a common build directory and manage
multiple targets.  This is the approach I would take to building multiple
executables from the same source files.

ghc doesn't do any locking of build files AFAIK.  Running parallel ghc
commands for two main modules that have the same import, using the same
working directory, is not safe.  In pathological cases the two different
main modules may even generate different code *for the imported module*.
 This sort of situation can arise with the IncoherentInstances extension,
for example.

The obvious approach is of course to make a library out of your common
files.  This has the downsides of requiring a bit more work on the
developer's part, but if the common files are relatively stable it'll
probably lead to the fastest builds of your executables.  Also in this case
you could run multiple `ghc --make`s in parallel, using different build
directories, since they won't be rebuilding any common code.

John L.

On Sun, Jan 5, 2014 at 1:47 PM, Sami Liedes <sami.liedes at iki.fi> wrote:

> Hi,
>
> I have a Haskell project where a number of executables are produced
> from mostly the same modules. I'm using a Makefile to enable parallel
> builds. I received advice[1] that ghc -M is broken, but that there
> is parallel ghc --make in HEAD.
>
> As far as I can tell, ghc --make does not allow building several
> binaries in one run, so I think it may not still be a full replacement
> for Makefiles.
>
> However I have a question about ghc --make that is also relevant
> without parallel ghc --make:
>
> If I have two main modules, prog1.hs and prog2.hs, which have mutual
> dependencies (for example, both import A from A.hs), is it safe to run
> "ghc --make prog1" in parallel with "ghc --make prog2"? IOW, is there
> some kind of locking to prevent both from building module A at the
> same time and interfering with each other?
>
> Is there a good way (either in current releases or HEAD) to build
> multiple binaries partially from the same sources in parallel?
>
>         Sami
>
>
> [1]
> http://stackoverflow.com/questions/20938894/generating-correct-link-dependencies-for-ghc-and-makefile-style-builds
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20140105/28e8e5f3/attachment.html>