[web-devel] simple-sendfile

[Michael Snoyman]

On Wed, Jun 15, 2011 at 4:43 AM, Kazu Yamamoto <kazu at iij.ad.jp> wrote:
> Hello,
>
> I have uploaded the simple-sendfile package to HackageDB.
>
>        http://hackage.haskell.org/package/simple-sendfile
>
> This is a simple version of the sendfile package which use unnecessary
> system calls (stat() and lseek()).
>
> The simple-sendfile package provides native sendfile support for
> Linux, BSDs, and MacOS. For other OSes, a fallback function using
> read()/send () is provided.
>
> Since the limitation of sendfile on Linux, stat() is called if
> EntireFile is used. But we can use PartOfFile instead to send entire
> files.
>
> I modified Warp to make use of the simple-sendfile package.
>
>        https://github.com/kazu-yamamoto/warp/commit/fd73ffb098475255756f7706f8328513643ed20c
>
> And I confirmed that performance becomes much better. I'm very
> satisfied with the current performance.
>
> Since I don't know how T.Handle should be used, the commit above just
> ignores T.Handle at this moment. If necessary, I would love to modify
> the API of the simple-sendfile package.
>
> --Kazu
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel
>

Hi Kazu,

Looks good, I'd love to merge this. I think the only thing we'd need
modified to the simple-sendfile API is to accept a callback that's
triggered each time a chunk is sent over the stream. How about:

    sendfile :: Socket -> FilePath -> FileRange -> IO () -> IO ()

Though it seems that you currently aren't doing any chunking, so
there's no way to tickle the timeout. This will work fine for small
files, but sending 2GB files will result in timeouts, unless we
disable timeouts for all sendfile calls. But of course that opens us
to slowloris attacks.

I know it will hurt performance a bit, but what do you think of
changing simple-sendfile to send in smaller chunks?

Michael