[web-devel] Making throughput much better

[Kazu Yamamoto ( 山本和彦 )]

Hi,

I confirmed that the throughput of acme-http is also improved:

No yield:  60,772 req/s
Yield:    133,478 req/s

I will try the internal-state branch of Warp next.

--Kazu

> On Mon, Jul 22, 2013 at 11:28 AM, Kazu Yamamoto <kazu at iij.ad.jp> wrote:
> 
>> Hi all,
>> Cc: Simon Marlow
>>
>> I think I found a way to make the throughput of web servers much
>> better. In short, put "Control.Concurrent.yield" after sending
>> something. This enables good schedule shaping.
>>
>> Typical code for web servers is like this:
>>
>>         loop = do
>>             keepAlive <- receiveRequest
>>             sendResponse
>>             when keepAlive loop
>>
>> With this code, a typical sequence of system calls is like this:
>>
>>         recvfrom(13, )                -- thread A
>>         sendto(13, )                  -- thread A
>>         recvfrom(13, ) = -1 EAGAIN    -- thread A
>>         epoll_ctl(3, )                -- IO manager
>>         recvfrom(14, )                -- thread B
>>         sendto(14, )                  -- thread B
>>         recvfrom(14, ) = -1 EAGAIN    -- thread B
>>         epoll_ctl(3, )                -- IO manager
>>
>> Since each thread calls recvfrom() immediately after sendto(), the
>> possibility that recvfrom() can receive a request is low. This
>> involves the IO manager.
>>
>> To make the possibility higher, I put "yield" after sendResponse:
>>
>>         loop = do
>>             keepAlive <- receiveRequest
>>             sendResponse
>>             yield
>>             when keepAlive loop
>>
>> Yield pushes its Haskell thread onto the end of thread queue. So,
>> another thread can work. During the work of other threads, a request
>> message would arrive.
>>
>>         recvfrom(13, )                -- thread A
>>         sendto(13, )                  -- thread A
>>         recvfrom(14, )                -- thread B
>>         sendto(14, )                  -- thread B
>>         recvfrom(13, )                -- thread A
>>         sendto(13, )                  -- thread A
>>
>> I tested this idea on SimpleServer and confirmed that its throughput
>> is doubled:
>>
>> No yield:  58,152 req/s (
>> https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs)
>> Yield:    113,048 req/s (
>> https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs)
>>
>> Unfortunately, yield does not work for Warp as expected. It does not
>> pass control to another thread and behaves as used to be.
>>
>> So, my question: why doesn't "yield" work in Warp? ResourceT is doing
>> a special thing? My code change is here:
>>
>>
>> https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c311f1
>>
>> I'm using GHC head (with multicore IO manager). I'm sure that the
>> following patch to fix the bug of yield is applied:
>>
>>
>> https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883
>>
>> P.S.
>>
>> It would be appreciated if someone tries this idea on other web
>> servers.
>>
>> --Kazu
>>
>> _______________________________________________
>> web-devel mailing list
>> web-devel at haskell.org
>> http://www.haskell.org/mailman/listinfo/web-devel
>>
> 
> Hey Kazu,
> 
> As usual, exciting work on the I/O manager.
> 
> I've had an experiment in the back of my mind for a while now, and after
> this email I decided to finally try it out. On the internal-state branch[1]
> of the WAI repo, I've implemented a relatively simple change:
> 
> * All operations that used to live in ResourceT IO now live in IO.
> * The Request value has a new field called resourceInternalState, which
> holds onto the internal state of the ResourceT transformer. Using the
> internal state API[2], it should be trivial to convert ResourceT code to
> live in the IO monad instead.
> 
> Would you be able to see if this change makes any meaningful impact on the
> benchmarks you're running? If so, we can discuss the proper way to make
> this transition. Note that, as I've implemented it, this is a breaking
> change in WAI, though we could take less extreme measures and get most of
> the performance benefit most likely.
> 
> Michael
> 
> [1] https://github.com/yesodweb/wai/tree/internal-state
> [2]
> http://hackage.haskell.org/packages/archive/resourcet/0.4.7.1/doc/html/Control-Monad-Trans-Resource.html#g:10