I/O manager: relying solely upon kqueue is not a safe way to go

[Andreas Voellmy]

Hi PHO,

Thanks for the detailed message. I will look at your report closely and get
back to you soon.

-Andi


On Fri, Mar 15, 2013 at 4:20 PM, Johan Tibell <johan.tibell at gmail.com>wrote:

> First, thanks for the very detailed investigation. I've CC:ed Andreas and
> Kazu who has the most knowledge of the I/O manager nowadays.
>
> -- Johan
>
> On Fri, Mar 15, 2013 at 12:54 PM, PHO <pho at cielonegro.org> wrote:
>
>> I found the HEAD stopped working on MacOS X 10.5.8 since the parallel
>> I/O manager got merged to HEAD. Stage-2 compiler successfully builds
>> (including Language.Haskell.TH.Syntax contrary to the report by Kazu
>> Yamamoto) but the resulting binary is very unstable especially for
>> ghci:
>>
>>   % inplace/bin/ghc-stage2  --interactive
>>   GHCi, version 7.7.20130313: http://www.haskell.org/ghc/  :? for help
>>   Loading package ghc-prim ... linking ... done.
>>   Loading package integer-gmp ... linking ... done.
>>   Loading package base ... linking ... done.
>>   Prelude>
>>   <stdin>: hGetChar: failed (Operation not supported)
>>
>> So I took a dtruss log and found it was kevent(2) that returned
>> ENOTSUP. GHC.Event.KQueue was just registering the stdin for
>> EVFILT_READ, whose type was of course tty, and then kevent(2) said
>> "tty is not supported". Didn't the old I/O manager do the same thing?
>> Why was it working then?
>>
>> After a hard investigation, I concluded that the old I/O manager was
>> not really working. It just looked fine but in fact wasn't. Here's an
>> explanation: If a fd to be registered is unsupported by kqueue,
>> kevent(2) returns -1 iff no incoming event buffer is passed
>> together. Otherwise it successfully returns with an incoming kevent
>> whose "flags" is EV_ERROR and "data" contains an errno. The I/O
>> manager has always been passing a non-empty event buffer until the
>> commit e5f5cfcd, while it wasn't (and still isn't) checking if a
>> received event in fact represents an error. That is, the KQueue
>> backend asks the kernel to monitor the stdin's readability. The kernel
>> then immediately delivers an event saying ENOTSUP. The KQueue backend
>> thinks "Hey, the stdin is now readable!" so it invokes a callback
>> associated with the fd. The thread which called "threadWaitRead" is
>> now awakened and performs a supposedly non-blocking read on the fd,
>> which in fact blocks but works anyway.
>>
>> However the situation has changed since the commit e5f5cfcd. The I/O
>> manager now registers fds without passing an incoming event buffer, so
>> kevent(2) no longer successfully delivers an error event instead it
>> directly returns -1 with errno set to ENOTSUP, hence the "Operation
>> not supported" exception.
>>
>> The Darwin's kqueue has started supporting tty since MacOS X 10.7
>> (Lion), but I heard it still doesn't support some important devices
>> like /dev/random. FreeBSD's kqueue has some difficulties too. It's no
>> doubt kqueue is a great mechanism for sockets, but IMHO it's not
>> something to use for all kinds of file I/O. Should we then try
>> kqueue(2) first and fallback to poll(2) if failed? Sadly no. Darwin's
>> poll(2) is broken too, and select(2) is the only method reliable:
>> http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#OS_X_AND_DARWIN_BUGS
>>
>> I wrote a small program to test if a given stdin is supported by
>> kqueue, poll and select:
>> https://gist.github.com/phonohawk/5169980#file-kqueue-poll-select-cpp
>>
>> MacOS X 10.5.8 is hopelessly broken. We can't use anything other than
>> select(2) for tty and other devices:
>>
>> https://gist.github.com/phonohawk/5169980#file-powerpc-apple-darwin9-8-0-txt
>>
>> FreeBSD 8.0 does support tty but not /dev/random. I don't know what
>> about the latest FreeBSD 9.1:
>> https://gist.github.com/phonohawk/5169980#file-i386-unknown-freebsd8-0-txt
>>
>> NetBSD 6.99.17 works perfectly here:
>>
>> https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-netbsd6-99-17-txt
>>
>> Just for reference, Linux 2.6.16 surely doesn't have kqueue(2) but it
>> supports poll(2)ing on devices:
>>
>> https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-linux2-6-16-txt
>>
>> I accordingly suggest that we should have some means to run two
>> independent I/O managers on BSD-like systems: KQueue for sockets,
>> pipes and regular file reads, and Select for any other devices and
>> regular file writes. Note also that no implementations of kqueue
>> support monitoring writability of regular file writes, which also
>> endangers the current use of kqueue-based I/O manager namely
>> "threadWaitWrite".
>>
>> Any ideas?
>>
>> Thanks,
>> PHO
>> _______________________________________________________
>>  - PHO -                         http://cielonegro.org/
>> OpenPGP public key: 1024D/1A86EF72
>> Fpr: 5F3E 5B5F 535C CE27 8254  4D1A 14E7 9CA7 1A86 EF72
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130315/26f8f9a1/attachment.htm>