[Haskell-cafe] Re: sendfile leaking descriptors on Linux?

[Bardur Arantsson]

Felipe Lessa wrote:
> On Sat, Feb 06, 2010 at 09:16:35AM +0100, Bardur Arantsson wrote:
>> Brandon S. Allbery KF8NH wrote:
>>> On Feb 5, 2010, at 02:56 , Bardur Arantsson wrote:
>> [--snip--]
>>> "Broken pipe" is normally handled as a signal, and is only mapped
>>> to an error if SIGPIPE is set to SIG_IGN.  I can well imagine that
>>> the SIGPIPE signal handler isn't closing resources properly; a
>>> workaround would be to use the System.Posix.Signals API to ignore
>>> SIGPIPE, but I don't know if that would work as a general solution
>>> (it would depend on what other uses of pipes/sockets exist).
>> It was a good idea, but it doesn't seem to help to add
>>
>> 	installHandler sigPIPE Ignore (Just fullSignalSet)
>>
>> to the main function. (Given the package name I assume
>> System.Posix.Signals works similarly to regular old signals, i.e.
>> globally per-process.)
>>
>> This is really starting to drive me round the bend...
> 
> Have you seen GHC ticket #1619?
> 
> http://hackage.haskell.org/trac/ghc/ticket/1619
> 
> 

I hadn't. I guess the conclusion is that SIG_PIPE is ignored by default anyway. So much
for that.

During yet another bout of debugging, I've added even more "I am here" instrumentation
code to the SendFile code, and the culprit seems to be threadWaitWrite. Here's the bit
of code I've modified:

 > sendfile :: Fd -> Fd -> Ptr Int64 -> Int64 -> IO Int64
 > sendfile out_fd in_fd poff bytes = do
 >     putStrLn "PRE-threadWaitWrite"
 >     threadWaitWrite out_fd
 >     putStrLn "AFTER threadWaitWrite"
 >     sbytes <- c_sendfile out_fd in_fd poff (fromIntegral bytes)
 >     putStrLn $ "AFTER c_sendfile; result was: " ++ (show sbytes)
 >     if sbytes <= -1
 >       then do errno <- getErrno
 >               if errno == eAGAIN
 >                 then sendfile out_fd in_fd poff bytes
 >                 else throwErrno "Network.Socket.SendFile.Linux"
 >       else return (fromIntegral sbytes)

This is the output when a file descriptor is lost:

---
AFTER sendfile: sbytes=27512
DIFFERENCE: 627264520
remaining=627264520, bytes=627264520
PRE-threadWaitWrite
Got request for CONTENT for objectId=1700000000000000,f215040000000000
Serving file 'X'...
Sending 625838080 bytes...
in_fd=13
---

So I have to conclude that threadWaitWrite is doing *something* which causes
the thread to die when the PS3 kills the connection.