[GHC] #8684: hWaitForInput cannot be interrupted by async exceptions on unix
GHC
ghc-devs at haskell.org
Fri Nov 17 16:18:05 UTC 2017
#8684: hWaitForInput cannot be interrupted by async exceptions on unix
-------------------------------------+-------------------------------------
Reporter: nh2 | Owner: (none)
Type: bug | Status: new
Priority: normal | Milestone:
Component: Core Libraries | Version: 7.6.3
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: 13497, 13525 | Blocking:
Related Tickets: #12912, #13525 | Differential Rev(s): Phab:D42
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by nh2):
I managed to make it work on Windows for `FILE_TYPE_CHAR`, by using
`WaitForMultipleObjects()` and passing it 2 objects: The `HANDLE` we're
interested in, an Event object that we signal when we want to interrupt
the thread.
After searching for a very long time, I ended up with the above solution.
Most important links:
* [https://social.msdn.microsoft.com/Forums/sqlserver/en-
US/dd7ce0e9-847d-4727-b0a6-efd68bd5626e/synchronous-readfile-on-stdin-
cannot-be-unblocked-by-cancelsynchronousio?forum=windowssdk Post by Ben
Golding in this thread] that proposes this solution
* [https://stackoverflow.com/questions/47336755/how-to-
cancelsynchronousio-on-waitforsingleobject-waiting-on-stdin My
StackOverflow question] (answers trickled in only after I had already
implemented a work-in-progress solution
Here's the backstory of me finding it out via the `#ghc` and `#winapi` IRC
channels:
In `#ghc`:
{{{
<nh2[m]> JaffaCake_: may I steal your attention for a minute? `ccall
interruptible` doesn't seem to work for me on Windows, it claims that
there's nothing to cancel, do you have an idea what this could be?
https://ghc.haskell.org/trac/ghc/ticket/8684#comment:25
<JaffaCake_> no idea, sorry
<JaffaCake_> there was a bug reported recently in process that also looks
like foreign ccall interruptible not working on windows
<nh2[m]> JaffaCake_: yes, I'm aware of it and have talked a bit with
Neil about it, I'll try to make that interruptible after I've made
hWaitForInput interruptible (already works on Linux -threaded for me), but
currently it seems interruptible doesn't work at all for me on Windows
<JaffaCake_> TBH I don't remember whether it ever worked
<nh2[m]> JaffaCake_: there's another thing that I've already looked
into a couple hours but haven't found a way yet:
<nh2[m]> My non-Windows fix works nicely on -threaded, but for
nonthreaded it doesn't, because the scheduler makes it immediately re-
enter the foreign call, so the `throwTo` in `timeout` doesn't get a chance
to run.
<nh2[m]> Would it be possible to say, in the scheduler: Whenever a
thread returns from a foreign call, yield, so that other Haskell threads
have a chance to run?
<JaffaCake_> I think that's probably a bad idea
<JaffaCake_> if the timeout has fired, then there should be a blocked
exception
<JaffaCake_> and that should get thrown immediately the FFI call returns
<nh2[m]> hWaitForInput/fdReady() gets woken up by the SIGVTALRM timer
signal, and then the scheduler immediately runs fdReady() again
<nh2[m]> so `timeout` never actually gets a chance to throwTo the
exception
<nh2[m]> so in other words, the timeout can never fire
<JaffaCake_> nh2[m], you're not running with -threaded?
<nh2[m]> JaffaCake_: no, that's what I meant, it's working fine with
-threaded on Linux, but not with non-threaded. This problem (and my
question on whether we could yield after FFI return) is only for non-
threaded
<JaffaCake_> oh, I'm not worried about non-threaded
<nh2[m]> JaffaCake_: but I am :D I'm trying to make it work equally
well across both threaded and non-threaded; in general I care a lot about
non-threaded as for some applications it is significantly faster, and it
is also easier to debug
<JaffaCake_> but you'll need to add hacks to make it work, hacks that
could add overhead for -threaded
<nh2[m]> JaffaCake_: my hope is that such overhead would be
negligible, as it would occur only for `interruptible` syscalls (which by
nature are expensive, so a bit of bookkeeping around them should not
matter much). For example, all calls to `fdReady()` that are performance-
sensitive, nonblocking (timeout=0), already use `unsafe` and thus wouldn't
suffer any overhead
<nh2[m]> having non-threaded not working would be quite a drag for
us, as we found it to perform way better for sequential-IO-heavy (e.g.
network) code
<nh2[m]> and with things like `timeout` not working for
`hWaitForInput`, we can't put timeouts on reads or writes in high-level
the code (would have to handle such logic at the very low level of
hWaitForInput itself)
<nh2[m]> e.g. `timeout N $ seriesOfVariousNetworkProtocolActions`
}}}
In `#winapi`:
{{{
nh2
18:54 hi, I'm trying to
WaitForSingleObject(GetStdHandle(STD_INPUT_HANDLE), ...) on stdin, and to
cancel this waiting using CancelSynchronousIo(), but the cancellation does
nothing (returns 0 and GetLastError() is ERROR_NOT_FOUND). Any idea what I
could be doing wrong?
SleepyISIS (IRC)
19:29 nh2[m]: If this function cannot find a request to cancel, the return
value is 0 (zero), and GetLastError returns ERROR_NOT_FOUND.
19:30 Are you sure that you want to cancel IO, not the wait itself?
nh2
19:31 SSleepyISIS: can you elaborate a bit on that distinction? To me
those sound like the same things
19:33 what I need to do is use a function that blocks until input is
available on the given HANDLE to a FILE_TYPE_CHAR, up to a maximum amount
of time (e.g. in milliseconds), but if some other event occurs, I need to
cancel that blocking wait
John___ (IRC)
19:33 nh2[m], there's SleepEx()
19:34 which does what you want.
19:35 also WaitForMultiple/SingleObject()
19:35 depends what your goal is.
nh2
19:35 JJohn___: that one seems to just wait a given amount of time,
independent from input being available on some HANDLE. I want to wait
until either the specified time has passed, or there's input available on
the HANDLE, or the waiting has been cancelled (e.g. using
CancelSynchronousIO)
SleepyISIS (IRC)
19:35 nh2[m]: Is terminating a thread an option?
John___ (IRC)
19:36 nh2[m], then WaitForSingleObject()
nh2
19:37 JJohn___: yes, that is what I am using, and what doesn't seem to
work (as I wrote above)
SleepyISIS (IRC)
19:37 Or stick to asynch solution.
John___ (IRC)
19:37 it waits for a timeout or when input is available on a handle.
19:37 I was not here to see what the problem was. mind pasting it again?
nh2
19:37 SSleepyISIS: regarding terminating a thread, likely not, it should
ideally work in a single-threaded use case (I'm doing this for an
improvement in the GHC Haskell compiler), but I would still very much like
to hear what you have in mind, maybe it is possible
SleepyISIS (IRC)
19:38 John___: If I got nh2[m] correctly, (s)he wants to abort synch IO
before the wait timeouts.
nh2
19:38 JJohn___: I've just written up the problem more clearly on
https://stackoverflow.com/questions/47336755/how-to-cancelsynchronousio-
on-waitforsingleobject-waiting-on-stdin
How to CancelSynchronousIo() on WaitForSingleObject() waiting on stdin?
On Windows 10, I'm waiting for input from the console using
WaitForSingleObject( GetStdHandle(STD_INPUT_HANDLE), ... ) and to cancel
this waiting using CancelSynchronousIo(). But the cancellatio...
SleepyISIS (IRC)
19:39 nh2[m]: https://stackoverflow.com/questions/34372611/how-to-signal-
file-handle-waiting-with-waitforsingleobject
How to signal file HANDLE waiting with WaitForSingleObject
This code, which I have no control over, reads a file using overlapped
I/O: // Read file asynchronously HANDLE hFile = CreateFile(...,
FILE_FLAG_OVERLAPPED, ...); BYTE buffer[10]; OVERLAPPED oRead...
19:39 Hmm, what value do you get from GetStdHandle?
John___ (IRC)
19:40 nh2[m], synchronous solutions cannot be "unblocked"
19:40 that's why you use async.
19:40 let me read the stack post first though before I reply differently.
nh2
19:42 JJohn___: it was my understanding so far that
CancelSynchronousIo()'s purpose is to cancel specifically functions like
WaitForSingleObject(), so I'm surprised that it doesn't work
John___ (IRC)
19:42 I don't think it's for those purposes.
nh2
19:42 SSleepyISIS: the HANDLE returned from GetStdHandle() is 0 for the
stdin
SleepyISIS (IRC)
19:43 >My colleague suggested WaitForMultipleObjects() as a workaround. So
we can wait on either the console/stdin handle or an event which signals
termination.
19:43 Good solution.
19:43 0 doesn't seem to be a valif handle.
19:44 Yeah: If an application does not have associated standard handles,
such as a service running on an interactive desktop, and has not
redirected them, the return value is NULL.
19:44 https://docs.microsoft.com/en-us/windows/console/getstdhandle
John___ (IRC)
19:44 yeah, that ^
19:45 using that in combination with SetEvent()
19:45 allows you to either make WaitForMultipleObjects() to catch on
WAIT_SIGNALED or WAIT_TIMEOUT
nh2
19:45 SSleepyISIS: oh sorry, I typoed my printf. The HANDLE returned from
GetStdHandle() is 84 for the stdin
John___ (IRC)
19:45 or whatevet
19:45 so you can make it either trigger when the handle input is available
nh2
19:48 SSleepyISIS: JJohn___ OK, so I would get rid of the
CancelSynchronousIo() completely, and always make sure that any
WaitForSingleObject() on stdin is instead a WaitForMultipleObjects() on
stdin AND the special separate event handle I use for "early termination"
of the wait?
Mysoft (IRC)
19:50 nh2[m]
19:50 you tried to use CancelIoEx
19:50 isntead of CancelIoSynchronous?
nh2
19:55 MMysoft: I will try that now; what would you pass as the
lpOverlapped argument, NULL?
Mysoft (IRC)
19:56 yeah
19:56 and which OS you are trying that?
19:56 and you're running from console?
19:56 or inside an IDE?
nh2
19:57 MMysoft: that call fails with GetLastError() == 6
(ERROR_INVALID_HANDLE). I'm on Windows 10, running from cmd.exe
Mysoft (IRC)
19:58 ok
19:58 so this was a behavior change
19:58 because here it works
19:58 i think it changes in some update on win7
19:58 (bug?)
19:59 there's a OldNewThing about the subject
19:59 https://blogs.msdn.microsoft.com/oldnewthing/20150323-00/?p=44413
20:00 but anyway using WaitForMultipleObjects is the way to go
20:00 as i think using CancelIoEx... looks as bad as "ungetc" :P
nh2
20:04 MMysoft: ah wait, I am using CancelIoEx() wrong. For the
CancelSynchronousIo() I had to provide a thread handle as an argument, but
for CancelIoEx() I have to provide the actual file handle as the argument.
Let me retry
20:05
nh2
20:06 MMysoft: OK, with that fixed, CancelIoEx() gives me GetLastError()
== ERROR_NOT_FOUND, like for CancelSynchronousIo()
20:07 I'll try the oldnewthing example code on my system
[Note by nh2: I didn't actually manage to make that compile immediately
with msys, and it didn't turn out to be necessary to solve the issue]
John___ (IRC)
20:46 nh2[m], yes, you pass 2 handles and you check which got signaled.
20:46 the result is WAIT_OBJECT_0 to (WAIT_OBJECT_0 + nCount– 1)
nh2
00:11 JJohn__: SSleepyISIS MMysoft : thanks for your help, I just got the
first working implementation for GHC that fixes its behaviour on Windows,
using your proposed extra event + WaitForMultipleObjects() approach!
}}}
Back in `#ghc`:
{{{
<nh2[m]> JaffaCake: I think I have finally figured out how to fix the
Windows problem with CancelSynchronousIo() having no effect. Some people
on #winapi helped me, saying that this function might not work in practice
and instead we could use WaitForMultipleObjects(), with 2 objects, one for
the actual FD HANDLE and one Event we'd signal if we want
WaitForMultipleObjects() to return early. I've coded that up and it works.
<nh2[m]> But for efficiency, I need one Event per thread, so that I
don't wake up all threads in `interruptWorkerTask()`, but only the target
one.
<nh2[m]> My plan is thus to have a `HANDLE interruptOSThreadEvent;`
in the `struct Task`
<nh2[m]> But I need to access that in `inputReady.c`, which is in
base, where I cannot get at the currently running task with `myTask()`
<nh2[m]> is there a way I can ask for the currently running Haskell
task from that location?
<JaffaCake> I'm pretty sure we do that in integer-gmp
<nh2[m]> JaffaCake: I don't know how thread-local storage actually
works internally, but I suppose it's done by symbol names somehow, and if
I just declare `extern __thread Task *my_task;` in `inputReady.c`, it
would refer to the right thing technically? (though I'd still have to
bring `Task` into scope)
<JaffaCake> just expose a function from the RTS to return the thread-
local Event
<JaffaCake> I was thinking of rts_unsafeGetMyCapability()
<JaffaCake> which is a similar kind of thing
<nh2[m]> JaffaCake: which would be the best header file to put it in?
Probably somewhere under `includes/rts`?
<JaffaCake> next to rts_unsafeGetMyCapability()
<nh2[m]> JaffaCake: OK, and it would have to go somewhere in
RtsSymbols.c, probably `RTS_MINGW_ONLY_SYMBOLS`, is that right?
}}}
--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8684#comment:28>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
More information about the ghc-tickets
mailing list