[Haskell-cafe] Re: sendfile leaking descriptors on Linux?

[Jeremy Shaw]

On Wed, Feb 17, 2010 at 2:36 AM, Taru Karttunen <taruti at taruti.net> wrote:

> Excerpts from Bardur Arantsson's message of Tue Feb 16 23:48:14 +0200 2010:
> > > This cannot be fixed in the sendfile library, it is a
> > > feature of TCP that connections may linger for a long
> > > time unless explicit timeouts are used.
> >
> > The problem is that the sendfile library *doesn't* wake
> > up when the connection is terminated (because of threadWaitWrite)
> > -- it doesn't matter what the timeout is.
>
> Even server code without sendfile has the same issue since
> all writing to sockets ends up using threadWaitWrite.
>

Right, this is my concern -- I want to make sure that all of happstack is
fixed, not just sendfile.


> System.Timeout.timeout terminates a threadWaitWrite using
> asynchronous exceptions.
>

So for sendfile, instead of threadWaitWrite we could do:

 r <- timeout (60 * 10^6) threadWaitWrite
 case r of
   Nothing -> ... -- timed out
   (Just ()) -> ... -- keep going

It seems tricky to use timeout at a higher level in the code, because some
requests may take a very long time to finish. For example, when serving a
long video, or streaming music it could be hours or days before the IO
request finishes.


If you want to detect dead sockets somewhat reliably
> without a timeout then there is SO_KEEPALIVE combined
> with polling SO_ERROR every few minutes.


 This approach sounds promising because it seems like it could be
incorporated into the guts of happstack-server. The timeout period could be
a Config option with a reasonable default. I would be surprised if *any*
happstack programs today are handling this correctly, so updating the core
to do something reasonable would be a big improvement... And if someone has
a special need where it is not ok, they can just change the config to use an
infinite timeout...

Does that sound like the right fix to you? (Obviously, if people are using
sendfile with something other than happstack, it does not help them, but it
sounds like trying to fix things in sendfile is misguided anyway.)

- jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100217/686c430d/attachment.html