[Haskell-cafe] Re: sendfile leaking descriptors on Linux?

[Bardur Arantsson]

Bardur Arantsson wrote:
> Bardur Arantsson wrote:
> 
> (sorry about replying-to-self)
> 
>> During yet another bout of debugging, I've added even more "I am here" 
>> instrumentation code to the SendFile code, and the culprit seems to be
>  > threadWaitWrite.
> 

As Jeremy Shaw pointed out off-list, the symptoms are also consistent
with a thread that simply gets stuck in threadWaitWrite.

I've tried a couple of different solutions to this based on starting a
separate thread to enforce a timeout on threadWaitWrite (using throwTo).

It seems to work to prevent the file descriptor leak, but causes GHC
to segfault after a while. Probably some sort of other resource exhaustion
since my code is just an evil hack:

 > killer :: MVar () -> ThreadId -> IO ()
 > killer dontKill otherThread = do
 >        threadDelay 5000
 >        x <- tryTakeMVar dontKill
 >        case x of
 >           Just _ -> putStrLn "Killer thread expired"
 >           Nothing -> throwTo otherThread (Overflow)

where the relevant bit of sendfile reads:

 >    mtid <- myThreadId
 >    dontKill <- newEmptyMVar
 >    forkIO $ killer dontKill mtid
 >    threadWaitWrite out_fd
 >    putMVar dontKill ()

So I'm basically creating a thread for every single "threadWaitWrite" operation
(which is a lot in this case).

Anyone got any ideas on a simpler way to handle this? Maybe I should just
report a bug for threadWaitWrite? IMO threadWaitWrite really should
throw some sort of IOException if the FD goes dead while it's waiting.

I suppose an alternative way to try to work around this would be by forcing the output
socket into blocking (as opposed to non-blocking) mode, but I can't figure out how to
do this on GHC 6.10.x -- I only see setNonBlockingFD which doesn't take a parameter
unlike its 6.12.x counterpart.

Cheers,