[web-devel] simple-sendfile v0.2.4

[Kazu Yamamoto ( 山本和彦 )]

Hello,

I have released simple-sendfile v0.2.4. In short, this version is 100+
times faster than v0.2.3 in some cases.

I always did benchmark mighttpd with high concurrency. That is, I
always make multiple connections at the same time. It gives me a good
result. However, when I set the number of concurrency to 1, I found
mighttpd is unbearablely slow on Linux and FreeBSD.

I realized that this is because that warp uses sendMany (the writev()
syscall) for a header and sendfile (the sendfile () syscall) for a
body (ie. a file). They are sent in separate TCP packets.

simple-sendfile v0.2.4 provides a new function called
sendfileWithHeader.

On Linux, it uses the send() syscall with the MSG_MORE flag to *store*
a header and the sendfile() syscall to send both the stored header and
a file. On FreeBSD, it uses the sendfile() syscall with the header
arguments to send both a header and a file. This trick ensures that
both a header and a body is sent in a single TCP packet.

On Mac, v0.2.3 sends separate packets but is fast. I don't know
why. v0.2.4 uses the same logic of FreeBSD. Unfortunately, the header
and the body is also sent separately but it is still fast. Again, I
don't know why.

Here is the result of benchmark:

         OLD+pronk  NEW+pronk  OLD+httperf  NEW+httperf
Mac      3678.74    3415.76    6642.5       6891.8
FreeBSD  10.0010    1309.43    10.0	    3174.8    # this machine is slow
Linux    24.9875    2558.36    25.0         8284.2

OLD means v0.2.3. NEW means v0.2.4.
pronk is "pronk -c 1 -n 1000 -r 10000 -b http://127.0.0.1:8000/"
httperf is "httperf --hog --num-conns 1 --num-calls 1000
                    --rate 10000 --server localhost --port 8000 --uri /"

I have already sent a pull request to warp to use sendfileWithHeader:
	https://github.com/yesodweb/wai/pull/86

P.S.

I also found that the network package cannot find IPV6_V6ONLY macro on
FreeBSD because the test code does not include "netinet/in.h". I will
sent a pull request to fix this later.

--Kazu