Parallel --make (GHC build times on newer MacBook Pros?)

[Simon Marlow]

On 01/09/2011 18:02, Evan Laforge wrote:
>>> It's an interesting idea that I hadn't thought of.  There would have to be
>>> an atomic file system operation to "commit" a compiled module - getting that
>>> right could be a bit tricky (compilation isn't deterministic, so the commit
>>> has to be atomic).
>>
>> I suppose you could just rename it into place when you're done.
>> -Edward
>
> I was imagining that it could create Module.o.compiling and then
> rename into place when it's done.  Then each ghc would do a work
> stealing thing where it tries to find output to produce that doesn't
> have an accompanying .compiling, or sleeps for a bit if all work at
> this stage is already taken, which is likely to happen since sometimes
> the graph would go through a bottleneck.  Then it's easy to clean up
> if work gets interrupted, just rm **/*.compiling

Right, using a Module.o.compiling file as a lock would work.

Another way to do this would be to have GHC --make invoke itself to 
compile each module separately.  Actually I think I prefer this method, 
although it might be a bit slower since each individual compilation has 
to read lots of interface files.  The main GHC --make process would do 
the final link only.  A fun hack for somebody?

Cheers,
	Simon