[GHC] #8400: Migrate the RTS to use libuv (or libev, or libevent)

[GHC]

#8400: Migrate the RTS to use libuv (or libev, or libevent)
----------------------------+----------------------------------------------
        Reporter:  schyler  |            Owner:  simonmar
            Type:  feature  |           Status:  new
  request                   |        Milestone:
        Priority:  normal   |          Version:
       Component:  Runtime  |         Keywords:
  System                    |     Architecture:  Unknown/Multiple
      Resolution:           |       Difficulty:  Project (more than a week)
Operating System:           |       Blocked By:
  Unknown/Multiple          |  Related Tickets:  635, 7353
 Type of failure:           |
  None/Unknown              |
       Test Case:           |
        Blocking:           |
----------------------------+----------------------------------------------

Comment (by thoughtpolice):

 I think there are separate discussions happening here. One is about moving
 to libuv because it provides some nice abstractions we don't have to roll
 ourselves, and it works on Windows - and the other is about moving the
 scheduler and related stuff around for better I/O performance. These are
 somewhat separate discussions, IMO.

 To move the I/O manager into the RTS (and port it from C to Haskell) would
 be a sizeable amount of complex work, everything else aside (it's much,
 much easier to get all the tricky multicore stuff right in Haskell,
 obviously!) Of course this shouldn't kill the suggestion, but it's worth
 bringing this up - it adds a sizeable amount of complexity to an already
 extremely complex thing (the RTS.) It also adds other tradeoffs. This
 isn't a thing we're new to - we often carry the complexity burden for
 users - but we also don't want to totally overblow our complexity budget.
 Frankly I think we should look into improvements beyond "totally rewrite
 it in C" - this should be a last resort only, unless some numbers would
 suggest huge order-of-magnitude improvements.

 For windows, #7353 discusses some of the issues with Joey's I/O manager,
 the main one being that it needs some sort of scheduler integration to
 help mitigate the performance loss from round-tripping through the kernel
 on every I/O request. I can definitely buy that scheduler integration may
 help latency numbers on all platforms. There's also the somewhat-related
 issue that `-threaded` always adds a bit of latency anyway, and the Mio
 paper touches on this. Perhaps we should do a more thorough investigation
 and experiments first. And also, Joey really just wanted interruptible
 socket timeouts. Perhaps as a first step, we should:

  A) investigate if Joey's patches in #7353 can be resurrected, perhaps as
 a library (his work happened before the introduction of Mio!) and

  B) see if there's performance on the table. There's probably some scope
 for this if we can find a talented Windows engineer to run some good
 numbers.

 If A) can happen at least, and the improvements are stable, I may not be
 opposed to integrating this I/O manager into `base`, provided Simon thinks
 it's OK. Although bad performance may be unfortunate, it also brings
 feature parity to windows in this area (which was woefully limited in
 other ways.)

 As for the other stuff libuv provides - we already tend to have very
 lightweight wrappers around most system concurrency primitives so I'm not
 sure how much we need it, but being maintained and less code for us is a
 win too. (But I don't find this to be too much of a selling point to us,
 personally. Just a unified interface to IOCP/epoll/kqueue is enough if we
 were to use it.)

 Not to deter anyone, I just find the discussion here touching on a few
 related things, and it's all quite a lot to consider and think about! I
 think there's a ton of scope for improvement here, but it seems quite open
 ended in terms of design and implementation. Performance numbers and
 patches welcome!

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8400#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler