[GHC] #7651: Buiding GHC with parallel IO manager freezes on Mac (not on FreeBSD)

[GHC]

#7651: Buiding GHC with parallel IO manager freezes on Mac (not on FreeBSD)
---------------------------------+------------------------------------------
    Reporter:  kazu-yamamoto     |       Owner:                     
        Type:  bug               |      Status:  new                
    Priority:  normal            |   Milestone:                     
   Component:  Build System      |     Version:  7.7                
    Keywords:                    |          Os:  MacOS X            
Architecture:  Unknown/Multiple  |     Failure:  Building GHC failed
  Difficulty:  Unknown           |    Testcase:                     
   Blockedby:                    |    Blocking:                     
     Related:                    |  
---------------------------------+------------------------------------------
Changes (by simonmar):

  * difficulty:  => Unknown


Old description:

> Building GHC with parallel IO manager on Mac freezes when
> compiling the dph libraries in the phase 2.
>
> * This happens only if we specify "-j" to "make". Note that "make" closes
> stdin of sub-processes if "-j" is specified.
>
> * Even if we specify "-j" to "make", the problem disappears with
> stdout/stderr redirection. That is, "make -jN >& LOG &" works.
>
> * The "-d" option of "make" does not make any effects.
>
> * Programs compiled with built GHC (with our patches) work well. For
> test, I compiled a daemon HTTP server which closes stdin/stdout/stderr.
> It worked well.
>
> An IO manager is polling a kqueue fd. Another Haskell thread on
> another native thread registers an event through the same kqueue
> fd.  In many cases, this works on Mac. In a certain situation,
> MacOS does not deliver an event to the IO manager. If the IO
> manager gets up and polls the kqueue fd, the event is delivered.
>
> This bug only appears when building GHC on Mac. I cannot find a
> simple way to reproduce it. Even if we find a way to reproduce
> it, I guess that we will probably reach a conclusion that this is
> a bug of kqueue of Mac.
>
> I have some evidences that kqueue of Mac is buggy:
>
> * http://comments.gmane.org/gmane.comp.lib.ev/1871
> * http://lists.apple.com/archives/darwin-dev/2011/Jun/msg00016.html

New description:

 Building GHC with parallel IO manager on Mac freezes when
 compiling the dph libraries in the phase 2.

 We '''suspect this is due to a bug in the OS X implementation of
 kqueue''', for the reasons given below.  In the meantime, we have added an
 extra IO manager wakeup that appears to work around the problem; see
 `GHC/Event/Manager.hs`.

 Details:

  * This happens only if we specify "-j" to "make". Note that "make" closes
 stdin of sub-processes if "-j" is specified.

  * Even if we specify "-j" to "make", the problem disappears with
 stdout/stderr redirection. That is, "make -jN >& LOG &" works.

  * The "-d" option of "make" does not make any effects.

  * Programs compiled with built GHC (with our patches) work well. For
 test, I compiled a daemon HTTP server which closes stdin/stdout/stderr. It
 worked well.

 An IO manager is polling a kqueue fd. Another Haskell thread on
 another native thread registers an event through the same kqueue
 fd.  In many cases, this works on Mac. In a certain situation,
 MacOS does not deliver an event to the IO manager. If the IO
 manager gets up and polls the kqueue fd, the event is delivered.

 This bug only appears when building GHC on Mac. I cannot find a
 simple way to reproduce it. Even if we find a way to reproduce
 it, I guess that we will probably reach a conclusion that this is
 a bug of kqueue of Mac.

 I have some evidences that kqueue of Mac is buggy:

  * http://comments.gmane.org/gmane.comp.lib.ev/1871
  * http://lists.apple.com/archives/darwin-dev/2011/Jun/msg00016.html

--

Comment:

 Added a bit of formatting and text to the description.

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7651#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler