[web-devel] Making throughput much better

[Kazu Yamamoto ( 山本和彦 )]

Hi all,
Cc: Simon Marlow

I think I found a way to make the throughput of web servers much
better. In short, put "Control.Concurrent.yield" after sending
something. This enables good schedule shaping.

Typical code for web servers is like this:

	loop = do
	    keepAlive <- receiveRequest
	    sendResponse
	    when keepAlive loop

With this code, a typical sequence of system calls is like this:

	recvfrom(13, )                -- thread A
	sendto(13, )                  -- thread A
	recvfrom(13, ) = -1 EAGAIN    -- thread A
	epoll_ctl(3, )                -- IO manager
	recvfrom(14, )                -- thread B
	sendto(14, )                  -- thread B
	recvfrom(14, ) = -1 EAGAIN    -- thread B
	epoll_ctl(3, )                -- IO manager

Since each thread calls recvfrom() immediately after sendto(), the
possibility that recvfrom() can receive a request is low. This
involves the IO manager.

To make the possibility higher, I put "yield" after sendResponse:

	loop = do
	    keepAlive <- receiveRequest
	    sendResponse
	    yield
	    when keepAlive loop

Yield pushes its Haskell thread onto the end of thread queue. So,
another thread can work. During the work of other threads, a request
message would arrive.

	recvfrom(13, )                -- thread A
	sendto(13, )                  -- thread A
	recvfrom(14, )                -- thread B
	sendto(14, )                  -- thread B
	recvfrom(13, )                -- thread A
	sendto(13, )                  -- thread A

I tested this idea on SimpleServer and confirmed that its throughput
is doubled:

No yield:  58,152 req/s (https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs)
Yield:    113,048 req/s (https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs)

Unfortunately, yield does not work for Warp as expected. It does not
pass control to another thread and behaves as used to be.

So, my question: why doesn't "yield" work in Warp? ResourceT is doing
a special thing? My code change is here:

	https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c311f1

I'm using GHC head (with multicore IO manager). I'm sure that the
following patch to fix the bug of yield is applied:

	https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883

P.S.

It would be appreciated if someone tries this idea on other web
servers.

--Kazu