Parallel building multiple targets
Simon Marlow
marlowsd at gmail.com
Thu Jan 23 09:28:36 UTC 2014
On 23/01/14 03:52, John Lato wrote:
> However, these are all rather obviously fixable as part of the build
> system. For me, the worst problems have to do with cleaning. If you're
> using a Makefile, typically you want to leave intermediate object files
> around and only rebuild them when the sources have changed. However,
> there are various issues with ghc batch-mode that make this difficult
> (e.g. https://ghc.haskell.org/trac/ghc/ticket/8029 ). The workarounds
> to deal with this are not as straightforward. The alternative is to
> live with the occasional build error that can only be fixed by blowing
> away the entire build dir (a remedy that I often need with ghc's source
> tree, as even make maintainer-clean doesn't always cut it. Hopefully my
> experience here is unique, but I do not believe it is).
You said "various issues", but you've only mentioned *one* specific
issue so far: #8029, and we concluded that was not a bug, although I do
see how it could require manually deleting a .hi file if you have a
module that shadows a package module and then remove it. This seems a
rare occurrence to me, but perhaps it is something you do often. If it
really hurts, then you could have a way to tell your build system about
a file when it is removed from the project, so that it can delete the
build artifacts that go with it.
Anyway, are there other problems you'd like to bring to our attention?
Cheers,
Simon
> Also, the most common use case seems to be for parallel building of
> modules. As ghc-7.8 provides this with --make, I'd expect the demand
> for ghc -M will be greatly reduced. That's why I'm not certain it's
> worth the time it would take to resolve these issues.
>
> Cheers,
> John
>
>
> Cheers,
> Simon
>
>
>
> For an example of some of the extra steps necessary to make
>
> something like this work, see e.g.
> https://github.com/nh2/__multishake
> <https://github.com/nh2/multishake> (which is admittedly for a more
> complicated setup, and also has some issues). The especially
> frustrating part is, just when you think you have everything
> working,
> someone wants to add some other tool to a workflow (hsc2hs, .cmm
> files,
> etc), and your build system doesn't support it.
>
> ghc --make doesn't allow building several binaries in one run,
> however
> if you use cabal all the separate runs will use a shared build
> directory, so subsequent builds will be able to take advantage
> of the
> intermediate output of the first build. Of course you could do
> the same
> without cabal, but it's a convenient way to create a common build
> directory and manage multiple targets. This is the approach I would
> take to building multiple executables from the same source files.
>
> ghc doesn't do any locking of build files AFAIK. Running
> parallel ghc
> commands for two main modules that have the same import, using
> the same
> working directory, is not safe. In pathological cases the two
> different
> main modules may even generate different code *for the imported
> module*.
> This sort of situation can arise with the IncoherentInstances
> extension, for example.
>
> The obvious approach is of course to make a library out of your
> common
> files. This has the downsides of requiring a bit more work on the
> developer's part, but if the common files are relatively stable
> it'll
> probably lead to the fastest builds of your executables. Also
> in this
> case you could run multiple `ghc --make`s in parallel, using
> different
> build directories, since they won't be rebuilding any common code.
>
> John L.
>
> On Sun, Jan 5, 2014 at 1:47 PM, Sami Liedes <sami.liedes at iki.fi
> <mailto:sami.liedes at iki.fi>
> <mailto:sami.liedes at iki.fi <mailto:sami.liedes at iki.fi>>> wrote:
>
> Hi,
>
> I have a Haskell project where a number of executables are
> produced
> from mostly the same modules. I'm using a Makefile to
> enable parallel
> builds. I received advice[1] that ghc -M is broken, but
> that there
> is parallel ghc --make in HEAD.
>
> As far as I can tell, ghc --make does not allow building
> several
> binaries in one run, so I think it may not still be a full
> replacement
> for Makefiles.
>
> However I have a question about ghc --make that is also
> relevant
> without parallel ghc --make:
>
> If I have two main modules, prog1.hs and prog2.hs, which
> have mutual
> dependencies (for example, both import A from A.hs), is it
> safe to run
> "ghc --make prog1" in parallel with "ghc --make prog2"?
> IOW, is there
> some kind of locking to prevent both from building module A
> at the
> same time and interfering with each other?
>
> Is there a good way (either in current releases or HEAD) to
> build
> multiple binaries partially from the same sources in parallel?
>
> Sami
>
>
> [1]
> http://stackoverflow.com/__questions/20938894/generating-__correct-link-dependencies-for-__ghc-and-makefile-style-builds
> <http://stackoverflow.com/questions/20938894/generating-correct-link-dependencies-for-ghc-and-makefile-style-builds>
>
> _________________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.__org
> <mailto:Glasgow-haskell-users at haskell.org>
> <mailto:Glasgow-haskell-users at __haskell.org
> <mailto:Glasgow-haskell-users at haskell.org>>
> http://www.haskell.org/__mailman/listinfo/glasgow-__haskell-users <http://www.haskell.org/mailman/listinfo/glasgow-haskell-users>
>
>
>
>
>
> _________________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.__org
> <mailto:Glasgow-haskell-users at haskell.org>
> http://www.haskell.org/__mailman/listinfo/glasgow-__haskell-users <http://www.haskell.org/mailman/listinfo/glasgow-haskell-users>
>
>
More information about the Glasgow-haskell-users
mailing list