[web-devel] Performance of Yesod 0.7 fourteen times slower than Rails 2.3?

Mon Mar 28 14:54:07 CEST 2011

Hi,

Just curies,
What does "79 times slower" mean? Does it mean that when Rails-dev has
processed 1 transaction Yesod did 79?

Best regards.

BTW: Saying that RoR is 79 times slower would indicate that Yesod is slow,
but RoR is (79 times) slower.

On 28 March 2011 14:31, Sven Koschnicke (GFXpro) <s.koschnicke at gfxpro.com>wrote:

> Hello,
>
> I tested again with persistent-postgresql 0.4.0.1 and now the results are
> what you would expect :) rails in dev mode is now 79 times slower[1]. Of
> course, thats not a fair compare, but Greg did some testing with rails
> production already. Thanks for improving that so quickly!
>
> Greetings
>  Sven
>
> [1] https://github.com/SKoschnicke/performance-test
>
>
> On 03/25/2011 08:41 AM, Michael Snoyman wrote:
>
>> Hi Sven,
>>
>> Can I ask you to try things out one more time, this time with
>> persistent-postgresql 0.4.0.1? I think I fixed some more of the
>> issue[1].
>>
>> Thanks,
>> Michael
>>
>> [1] http://docs.yesodweb.com/blog/improving-persistent-performance
>>
>> On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)
>> <s.koschnicke at gfxpro.com>  wrote:
>>
>>> Great that you found a way to speed things up! I'm looking forward to the
>>> blog post. I will run some more tests with the improved code, just let me
>>> know when its available. Is someone working on improving the postgres
>>> database backend? Maybe I could help there, the bad performance with
>>> postgres is really a showstopper for me.
>>>
>>> Sven
>>>
>>> On 03/23/2011 08:13 AM, Michael Snoyman wrote:
>>>
>>>>
>>>> This was a great example of a performance bug being exactly where you
>>>> don't expect it. I'm planning on writing up a blog post on this
>>>> adventure later, but for the quick summary: the Criterion benchmarks
>>>> have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200
>>>> fold speedup for some operations. This came from switching a large
>>>> monad transformer stack to a single RWS monad. But more on that in the
>>>> blog post.
>>>>
>>>> I've run your test a few more times: it still seems like there's a
>>>> slowdown from the database code, but I think the widget performance
>>>> bug was the big issue. After some more testing on my end, I'll put the
>>>> code on Hackage. Would you give the test another shot after that?
>>>>
>>>> Thanks again for bringing this up!
>>>>
>>>> Michael
>>>>
>>>> On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)
>>>> <s.koschnicke at gfxpro.com>    wrote:
>>>>
>>>>>
>>>>> Hello,
>>>>>
>>>>> I think your guess wasn't completely wrong. I tested with Sqlite now,
>>>>> and
>>>>> with that rails and yesod are equally slow (rails 11.81 secs vs yesod
>>>>> 11.36
>>>>> secs). But its also interesting that your code change speeds it up.
>>>>>
>>>>> (I put the sqlite benchmark at
>>>>> https://github.com/SKoschnicke/performance-test/tree/sqlite)
>>>>>
>>>>> My first guess was actually that its the marshalling which slows yesod
>>>>> so
>>>>> much and that rails uses some clever optimizations there, but I haven't
>>>>> got
>>>>> enough insight to validate that.
>>>>>
>>>>> Greetings
>>>>>  Sven
>>>>>
>>>>>
>>>>>
>>>>> On 03/21/2011 01:49 PM, Michael Snoyman wrote:
>>>>>
>>>>>>
>>>>>> Looks like my initial guess was completely wrong: on my system, almost
>>>>>> no time is spent on the database query. Instead, it looks like there's
>>>>>> some kind of performance issue with Hamlet. For example, changing line
>>>>>> 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases
>>>>>> performance dramatically (25 sec to 4.5 sec). I'll need to spend some
>>>>>> more time to properly diagnose the problem. Again, thanks for bringing
>>>>>> this to my attention.
>>>>>>
>>>>>> Michael
>>>>>>
>>>>>> On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman<michael at snoyman.com
>>>>>> >
>>>>>>  wrote:
>>>>>>
>>>>>>>
>>>>>>> Firstly, thank you for running this benchmark: it's the only way we
>>>>>>> can find out where Yesod needs to be fixed.
>>>>>>>
>>>>>>> I haven't had a chance to look into the code properly, but it looks
>>>>>>> like this is more a test of the database backend than of the web
>>>>>>> framework. I'll try to get to this myself later, but it would be very
>>>>>>> interesting to see the difference in response times between
>>>>>>> PostgreSQL
>>>>>>> and sqlite. Currently, persistent-sqlite binds directly to the sqlite
>>>>>>> C API, while persistent-postgresql uses HDBC. It's entirely possible
>>>>>>> that HDBC is adding some overhead.
>>>>>>>
>>>>>>> One of the items on our wish list[1] is to migrate away from HDBC.
>>>>>>> Having hard numbers like you are providing will help us figure out
>>>>>>> our
>>>>>>> priorities a bit better.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Michael
>>>>>>>
>>>>>>> [1] http://wiki.yesodweb.com/Wishlist
>>>>>>>
>>>>>>> On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)
>>>>>>> <s.koschnicke at gfxpro.com>      wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I'm coming from a Rails background and was very impressed by the
>>>>>>>> features of
>>>>>>>> the Yesod Web Framework. Working on an trading app in Rails I wanted
>>>>>>>> to
>>>>>>>> port
>>>>>>>> some features of this app to a Yesod app just to learn the framework
>>>>>>>> and
>>>>>>>> because we have some performance issues with Rails.
>>>>>>>>
>>>>>>>> The first thing I noticed was that the Yesod app was significantly
>>>>>>>> slower
>>>>>>>> than the Rails app at trivial tasks. I am very astonished about
>>>>>>>> that.
>>>>>>>> I
>>>>>>>> thought Rails should be slower because of its interpreted nature. I
>>>>>>>> made
>>>>>>>> some benchmarking and measured that a simple Rails app is fourteen
>>>>>>>> times
>>>>>>>> faster than a Yesod app that did the same thing (loading some
>>>>>>>> records
>>>>>>>> from
>>>>>>>> the database and displaying them). Did I made a mistake or is my
>>>>>>>> understanding, that the compiled app should be faster, just wrong?
>>>>>>>>
>>>>>>>> I documented the results on github:
>>>>>>>>
>>>>>>>> https://github.com/SKoschnicke/performance-test
>>>>>>>>
>>>>>>>> Greetings
>>>>>>>>  Sven Koschnicke
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> web-devel mailing list
>>>>>>>> web-devel at haskell.org
>>>>>>>> http://www.haskell.org/mailman/listinfo/web-devel
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel
>

-- 
Knut Olav Bøhmer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/web-devel/attachments/20110328/d8c2906a/attachment-0001.htm>