Parallel building multiple targets

[Simon Marlow]

On 05/01/2014 23:48, John Lato wrote:
> (FYI, I expect I'm the source of the suggestion that ghc -M is broken)
>
> First, just to clarify, I don't think ghc -M is obviously broken.
>   Rather, I think it's broken in subtle, unobvious ways, such that
> trying to develop a make-based project with ghc -M will fail at various
> times in a non-obvious fashion, at least without substantial additional
> rules.

If I understand you correctly, you're not saying that ghc -M is broken, 
but that it would be easier to use if it did more.  Right?  Maybe you 
could make specific suggestions?  Saying it is "broken" is a bit 
FUD-ish.  We use it in GHC's build system, so by an existence proof it 
is certainly not broken.

Cheers,
Simon


   For an example of some of the extra steps necessary to make
> something like this work, see e.g.
> https://github.com/nh2/multishake (which is admittedly for a more
> complicated setup, and also has some issues).  The especially
> frustrating part is, just when you think you have everything working,
> someone wants to add some other tool to a workflow (hsc2hs, .cmm files,
> etc), and your build system doesn't support it.
>
> ghc --make doesn't allow building several binaries in one run, however
> if you use cabal all the separate runs will use a shared build
> directory, so subsequent builds will be able to take advantage of the
> intermediate output of the first build.  Of course you could do the same
> without cabal, but it's a convenient way to create a common build
> directory and manage multiple targets.  This is the approach I would
> take to building multiple executables from the same source files.
>
> ghc doesn't do any locking of build files AFAIK.  Running parallel ghc
> commands for two main modules that have the same import, using the same
> working directory, is not safe.  In pathological cases the two different
> main modules may even generate different code *for the imported module*.
>   This sort of situation can arise with the IncoherentInstances
> extension, for example.
>
> The obvious approach is of course to make a library out of your common
> files.  This has the downsides of requiring a bit more work on the
> developer's part, but if the common files are relatively stable it'll
> probably lead to the fastest builds of your executables.  Also in this
> case you could run multiple `ghc --make`s in parallel, using different
> build directories, since they won't be rebuilding any common code.
>
> John L.
>
> On Sun, Jan 5, 2014 at 1:47 PM, Sami Liedes <sami.liedes at iki.fi
> <mailto:sami.liedes at iki.fi>> wrote:
>
>     Hi,
>
>     I have a Haskell project where a number of executables are produced
>     from mostly the same modules. I'm using a Makefile to enable parallel
>     builds. I received advice[1] that ghc -M is broken, but that there
>     is parallel ghc --make in HEAD.
>
>     As far as I can tell, ghc --make does not allow building several
>     binaries in one run, so I think it may not still be a full replacement
>     for Makefiles.
>
>     However I have a question about ghc --make that is also relevant
>     without parallel ghc --make:
>
>     If I have two main modules, prog1.hs and prog2.hs, which have mutual
>     dependencies (for example, both import A from A.hs), is it safe to run
>     "ghc --make prog1" in parallel with "ghc --make prog2"? IOW, is there
>     some kind of locking to prevent both from building module A at the
>     same time and interfering with each other?
>
>     Is there a good way (either in current releases or HEAD) to build
>     multiple binaries partially from the same sources in parallel?
>
>              Sami
>
>
>     [1]
>     http://stackoverflow.com/questions/20938894/generating-correct-link-dependencies-for-ghc-and-makefile-style-builds
>
>     _______________________________________________
>     Glasgow-haskell-users mailing list
>     Glasgow-haskell-users at haskell.org
>     <mailto:Glasgow-haskell-users at haskell.org>
>     http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
>
>
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>