[web-devel] Making throughput much better

[Michael Snoyman]

On Mon, Jul 22, 2013 at 11:28 AM, Kazu Yamamoto <kazu at iij.ad.jp> wrote:

> Hi all,
> Cc: Simon Marlow
>
> I think I found a way to make the throughput of web servers much
> better. In short, put "Control.Concurrent.yield" after sending
> something. This enables good schedule shaping.
>
> Typical code for web servers is like this:
>
>         loop = do
>             keepAlive <- receiveRequest
>             sendResponse
>             when keepAlive loop
>
> With this code, a typical sequence of system calls is like this:
>
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>         recvfrom(13, ) = -1 EAGAIN    -- thread A
>         epoll_ctl(3, )                -- IO manager
>         recvfrom(14, )                -- thread B
>         sendto(14, )                  -- thread B
>         recvfrom(14, ) = -1 EAGAIN    -- thread B
>         epoll_ctl(3, )                -- IO manager
>
> Since each thread calls recvfrom() immediately after sendto(), the
> possibility that recvfrom() can receive a request is low. This
> involves the IO manager.
>
> To make the possibility higher, I put "yield" after sendResponse:
>
>         loop = do
>             keepAlive <- receiveRequest
>             sendResponse
>             yield
>             when keepAlive loop
>
> Yield pushes its Haskell thread onto the end of thread queue. So,
> another thread can work. During the work of other threads, a request
> message would arrive.
>
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>         recvfrom(14, )                -- thread B
>         sendto(14, )                  -- thread B
>         recvfrom(13, )                -- thread A
>         sendto(13, )                  -- thread A
>
> I tested this idea on SimpleServer and confirmed that its throughput
> is doubled:
>
> No yield:  58,152 req/s (
> https://gist.github.com/AndreasVoellmy/4184520#file-simpleserver-hs)
> Yield:    113,048 req/s (
> https://gist.github.com/kazu-yamamoto/6051118#file-simpleserver2-hs)
>
> Unfortunately, yield does not work for Warp as expected. It does not
> pass control to another thread and behaves as used to be.
>
> So, my question: why doesn't "yield" work in Warp? ResourceT is doing
> a special thing? My code change is here:
>
>
> https://github.com/yesodweb/wai/commit/4e9c6316ad59cd87be13000d15df8e6fd7c311f1
>
> I'm using GHC head (with multicore IO manager). I'm sure that the
> following patch to fix the bug of yield is applied:
>
>
> https://github.com/ghc/ghc/commit/66839957a376dbe85608822c1820eb6c99210883
>
> P.S.
>
> It would be appreciated if someone tries this idea on other web
> servers.
>
> --Kazu
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel
>

Hey Kazu,

As usual, exciting work on the I/O manager.

I've had an experiment in the back of my mind for a while now, and after
this email I decided to finally try it out. On the internal-state branch[1]
of the WAI repo, I've implemented a relatively simple change:

* All operations that used to live in ResourceT IO now live in IO.
* The Request value has a new field called resourceInternalState, which
holds onto the internal state of the ResourceT transformer. Using the
internal state API[2], it should be trivial to convert ResourceT code to
live in the IO monad instead.

Would you be able to see if this change makes any meaningful impact on the
benchmarks you're running? If so, we can discuss the proper way to make
this transition. Note that, as I've implemented it, this is a breaking
change in WAI, though we could take less extreme measures and get most of
the performance benefit most likely.

Michael

[1] https://github.com/yesodweb/wai/tree/internal-state
[2]
http://hackage.haskell.org/packages/archive/resourcet/0.4.7.1/doc/html/Control-Monad-Trans-Resource.html#g:10
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/web-devel/attachments/20130722/d53c4490/attachment.htm>