[web-devel] Making throughput much better

[Gregory Collins]

Hey Kazu,

I am planning to try this today, I think it will help benchmarks at the
very least. ;) I'll report my findings.

I suspect, however, that in a real server the improvement might be smaller
-- normally after sending a response a production server will do some other
IO (like access logging). I don't see where yielding would hurt, however.

G


On Mon, Jul 22, 2013 at 1:28 AM, Kazu Yamamoto <kazu at iij.ad.jp> wrote:

> Hi all,
> Cc: Simon Marlow
>
> I think I found a way to make the throughput of web servers much
> better. In short, put "Control.Concurrent.yield" after sending
> something. This enables good schedule shaping.
>
> Typical code for web servers is like this:
>
>         loop = do
>             keepAlive <- receiveRequest
>             sendResponse
>             when keepAlive loop
>
> With this code, a typical sequence of system calls is like this:
>
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>         recvfrom(13, ) = -1 EAGAIN    -- thread A
>         epoll_ctl(3, )                -- IO manager
>         recvfrom(14, )                -- thread B
>         sendto(14, )                  -- thread B
>         recvfrom(14, ) = -1 EAGAIN    -- thread B
>         epoll_ctl(3, )                -- IO manager
>
> Since each thread calls recvfrom() immediately after sendto(), the
> possibility that recvfrom() can receive a request is low. This
> involves the IO manager.
>
> To make the possibility higher, I put "yield" after sendResponse:
>
>         loop = do
>             keepAlive <- receiveRequest
>             sendResponse
>             yield
>             when keepAlive loop
>
> Yield pushes its Haskell thread onto the end of thread queue. So,
> another thread can work. During the work of other threads, a request
> message would arrive.
>
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>         recvfrom(14, )                -- thread B
>         sendto(14, )                  -- thread B
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>
> I tested this idea on SimpleServer and confirmed that its throughput
> is doubled:
>
> No yield:  58,152 req/s (
> https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs)
> Yield:    113,048 req/s (
> https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs)
>
> Unfortunately, yield does not work for Warp as expected. It does not
> pass control to another thread and behaves as used to be.
>
> So, my question: why doesn't "yield" work in Warp? ResourceT is doing
> a special thing? My code change is here:
>
>
> https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c311f1
>
> I'm using GHC head (with multicore IO manager). I'm sure that the
> following patch to fix the bug of yield is applied:
>
>
> https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883
>
> P.S.
>
> It would be appreciated if someone tries this idea on other web
> servers.
>
> --Kazu
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel
>



-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/web-devel/attachments/20130722/a070be27/attachment.htm>