Parallel --make (GHC build times on newer MacBook Pros?)

[Joachim Breitner]

Hi,

Am Freitag, den 02.09.2011, 09:07 +0100 schrieb Simon Marlow:
> On 01/09/2011 18:02, Evan Laforge wrote:
> >>> It's an interesting idea that I hadn't thought of.  There would have to be
> >>> an atomic file system operation to "commit" a compiled module - getting that
> >>> right could be a bit tricky (compilation isn't deterministic, so the commit
> >>> has to be atomic).
> >>
> >> I suppose you could just rename it into place when you're done.
> >> -Edward
> >
> > I was imagining that it could create Module.o.compiling and then
> > rename into place when it's done.  Then each ghc would do a work
> > stealing thing where it tries to find output to produce that doesn't
> > have an accompanying .compiling, or sleeps for a bit if all work at
> > this stage is already taken, which is likely to happen since sometimes
> > the graph would go through a bottleneck.  Then it's easy to clean up
> > if work gets interrupted, just rm **/*.compiling
> 
> Right, using a Module.o.compiling file as a lock would work.
> 
> Another way to do this would be to have GHC --make invoke itself to 
> compile each module separately.  Actually I think I prefer this method, 
> although it might be a bit slower since each individual compilation has 
> to read lots of interface files.  The main GHC --make process would do 
> the final link only.  A fun hack for somebody?

this would also help building large libraries on architectures with
little memory, as it seems to me that when one ghc instance is compiling
multiple modules in a row, some leaked memory/unevaluated thunks pile up
and eventually cause the compilation to abort. I suspect that building
each file on its own avoids this issue.

(But this is only based on observation, not on hard facts.)

Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
  mail at joachim-breitner.de  |  nomeata at debian.org  |  GPG: 0x4743206C
  xmpp: nomeata at joachim-breitner.de | http://www.joachim-breitner.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20110902/3775cd7e/attachment-0001.pgp>