[Haskell-cafe] Re: sendfile leaking descriptors on Linux?

[Jeremy Shaw]

It's not clear to me that this is actually a bug in threadWaitWrite.

I believe that under Linux, select() does not wakeup just because the file
descriptor was closed. (Under Windows, and possibly solaris/BSD/etc it
does). So this behavior might be consistent with normal Linux behavior.
However, it is clearly annoying that (a) the expected behavior is not
documented (b) the behavior might be different under Linux than other OSes.

In some sense it is correct -- if the file descriptor is closed, then we
certainly can't write more to it -- so threadWaitWrite need not wake up..
But that leaves us with the issue of needing  someway to be notified that
the file descriptor was closed so that we can clean up after ourselves..

- jeremy

On Sun, Feb 7, 2010 at 2:13 AM, Bardur Arantsson <spam at scientician.net>wrote:

> Bardur Arantsson wrote:
>
>> Bardur Arantsson wrote:
>>
>> (sorry about replying-to-self)
>>
>>  During yet another bout of debugging, I've added even more "I am here"
>>> instrumentation code to the SendFile code, and the culprit seems to be
>>>
>>  > threadWaitWrite.
>>
>>
> As Jeremy Shaw pointed out off-list, the symptoms are also consistent
> with a thread that simply gets stuck in threadWaitWrite.
>
> I've tried a couple of different solutions to this based on starting a
> separate thread to enforce a timeout on threadWaitWrite (using throwTo).
>
> It seems to work to prevent the file descriptor leak, but causes GHC
> to segfault after a while. Probably some sort of other resource exhaustion
> since my code is just an evil hack:
>
> > killer :: MVar () -> ThreadId -> IO ()
> > killer dontKill otherThread = do
> >        threadDelay 5000
> >        x <- tryTakeMVar dontKill
> >        case x of
> >           Just _ -> putStrLn "Killer thread expired"
> >           Nothing -> throwTo otherThread (Overflow)
>
> where the relevant bit of sendfile reads:
>
> >    mtid <- myThreadId
> >    dontKill <- newEmptyMVar
> >    forkIO $ killer dontKill mtid
> >    threadWaitWrite out_fd
> >    putMVar dontKill ()
>
> So I'm basically creating a thread for every single "threadWaitWrite"
> operation
> (which is a lot in this case).
>
> Anyone got any ideas on a simpler way to handle this? Maybe I should just
> report a bug for threadWaitWrite? IMO threadWaitWrite really should
> throw some sort of IOException if the FD goes dead while it's waiting.
>
> I suppose an alternative way to try to work around this would be by forcing
> the output
> socket into blocking (as opposed to non-blocking) mode, but I can't figure
> out how to
> do this on GHC 6.10.x -- I only see setNonBlockingFD which doesn't take a
> parameter
> unlike its 6.12.x counterpart.
>
>
> Cheers,
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100207/b6b7050c/attachment.html