[web-devel] Performance of Yesod 0.7 fourteen times slower than Rails 2.3?

[Sven Koschnicke (GFXpro)]

Hello Knut,

as stated in my last mail, it's not 79 but 7.5. And it means that in my 
benchmark[1], rails had an average response time of 0.53 seconds and 
yesod had 0.07 seconds.

Greetings
   Sven

[1] https://github.com/SKoschnicke/performance-test

On 03/28/2011 02:54 PM, Knut Olav Bøhmer wrote:
> Hi,
>
> Just curies,
> What does "79 times slower" mean? Does it mean that when Rails-dev has
> processed 1 transaction Yesod did 79?
>
> Best regards.
>
>
>
> BTW: Saying that RoR is 79 times slower would indicate that Yesod is slow,
> but RoR is (79 times) slower.
>
>
> On 28 March 2011 14:31, Sven Koschnicke (GFXpro)<s.koschnicke at gfxpro.com>wrote:
>
>> Hello,
>>
>> I tested again with persistent-postgresql 0.4.0.1 and now the results are
>> what you would expect :) rails in dev mode is now 79 times slower[1]. Of
>> course, thats not a fair compare, but Greg did some testing with rails
>> production already. Thanks for improving that so quickly!
>>
>> Greetings
>>   Sven
>>
>> [1] https://github.com/SKoschnicke/performance-test
>>
>>
>> On 03/25/2011 08:41 AM, Michael Snoyman wrote:
>>
>>> Hi Sven,
>>>
>>> Can I ask you to try things out one more time, this time with
>>> persistent-postgresql 0.4.0.1? I think I fixed some more of the
>>> issue[1].
>>>
>>> Thanks,
>>> Michael
>>>
>>> [1] http://docs.yesodweb.com/blog/improving-persistent-performance
>>>
>>> On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)
>>> <s.koschnicke at gfxpro.com>   wrote:
>>>
>>>> Great that you found a way to speed things up! I'm looking forward to the
>>>> blog post. I will run some more tests with the improved code, just let me
>>>> know when its available. Is someone working on improving the postgres
>>>> database backend? Maybe I could help there, the bad performance with
>>>> postgres is really a showstopper for me.
>>>>
>>>> Sven
>>>>
>>>> On 03/23/2011 08:13 AM, Michael Snoyman wrote:
>>>>
>>>>>
>>>>> This was a great example of a performance bug being exactly where you
>>>>> don't expect it. I'm planning on writing up a blog post on this
>>>>> adventure later, but for the quick summary: the Criterion benchmarks
>>>>> have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200
>>>>> fold speedup for some operations. This came from switching a large
>>>>> monad transformer stack to a single RWS monad. But more on that in the
>>>>> blog post.
>>>>>
>>>>> I've run your test a few more times: it still seems like there's a
>>>>> slowdown from the database code, but I think the widget performance
>>>>> bug was the big issue. After some more testing on my end, I'll put the
>>>>> code on Hackage. Would you give the test another shot after that?
>>>>>
>>>>> Thanks again for bringing this up!
>>>>>
>>>>> Michael
>>>>>
>>>>> On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)
>>>>> <s.koschnicke at gfxpro.com>     wrote:
>>>>>
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I think your guess wasn't completely wrong. I tested with Sqlite now,
>>>>>> and
>>>>>> with that rails and yesod are equally slow (rails 11.81 secs vs yesod
>>>>>> 11.36
>>>>>> secs). But its also interesting that your code change speeds it up.
>>>>>>
>>>>>> (I put the sqlite benchmark at
>>>>>> https://github.com/SKoschnicke/performance-test/tree/sqlite)
>>>>>>
>>>>>> My first guess was actually that its the marshalling which slows yesod
>>>>>> so
>>>>>> much and that rails uses some clever optimizations there, but I haven't
>>>>>> got
>>>>>> enough insight to validate that.
>>>>>>
>>>>>> Greetings
>>>>>>   Sven
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 03/21/2011 01:49 PM, Michael Snoyman wrote:
>>>>>>
>>>>>>>
>>>>>>> Looks like my initial guess was completely wrong: on my system, almost
>>>>>>> no time is spent on the database query. Instead, it looks like there's
>>>>>>> some kind of performance issue with Hamlet. For example, changing line
>>>>>>> 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases
>>>>>>> performance dramatically (25 sec to 4.5 sec). I'll need to spend some
>>>>>>> more time to properly diagnose the problem. Again, thanks for bringing
>>>>>>> this to my attention.
>>>>>>>
>>>>>>> Michael
>>>>>>>
>>>>>>> On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman<michael at snoyman.com
>>>>>>>>
>>>>>>>   wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Firstly, thank you for running this benchmark: it's the only way we
>>>>>>>> can find out where Yesod needs to be fixed.
>>>>>>>>
>>>>>>>> I haven't had a chance to look into the code properly, but it looks
>>>>>>>> like this is more a test of the database backend than of the web
>>>>>>>> framework. I'll try to get to this myself later, but it would be very
>>>>>>>> interesting to see the difference in response times between
>>>>>>>> PostgreSQL
>>>>>>>> and sqlite. Currently, persistent-sqlite binds directly to the sqlite
>>>>>>>> C API, while persistent-postgresql uses HDBC. It's entirely possible
>>>>>>>> that HDBC is adding some overhead.
>>>>>>>>
>>>>>>>> One of the items on our wish list[1] is to migrate away from HDBC.
>>>>>>>> Having hard numbers like you are providing will help us figure out
>>>>>>>> our
>>>>>>>> priorities a bit better.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Michael
>>>>>>>>
>>>>>>>> [1] http://wiki.yesodweb.com/Wishlist
>>>>>>>>
>>>>>>>> On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)
>>>>>>>> <s.koschnicke at gfxpro.com>       wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I'm coming from a Rails background and was very impressed by the
>>>>>>>>> features of
>>>>>>>>> the Yesod Web Framework. Working on an trading app in Rails I wanted
>>>>>>>>> to
>>>>>>>>> port
>>>>>>>>> some features of this app to a Yesod app just to learn the framework
>>>>>>>>> and
>>>>>>>>> because we have some performance issues with Rails.
>>>>>>>>>
>>>>>>>>> The first thing I noticed was that the Yesod app was significantly
>>>>>>>>> slower
>>>>>>>>> than the Rails app at trivial tasks. I am very astonished about
>>>>>>>>> that.
>>>>>>>>> I
>>>>>>>>> thought Rails should be slower because of its interpreted nature. I
>>>>>>>>> made
>>>>>>>>> some benchmarking and measured that a simple Rails app is fourteen
>>>>>>>>> times
>>>>>>>>> faster than a Yesod app that did the same thing (loading some
>>>>>>>>> records
>>>>>>>>> from
>>>>>>>>> the database and displaying them). Did I made a mistake or is my
>>>>>>>>> understanding, that the compiled app should be faster, just wrong?
>>>>>>>>>
>>>>>>>>> I documented the results on github:
>>>>>>>>>
>>>>>>>>> https://github.com/SKoschnicke/performance-test
>>>>>>>>>
>>>>>>>>> Greetings
>>>>>>>>>   Sven Koschnicke
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> web-devel mailing list
>>>>>>>>> web-devel at haskell.org
>>>>>>>>> http://www.haskell.org/mailman/listinfo/web-devel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>> _______________________________________________
>> web-devel mailing list
>> web-devel at haskell.org
>> http://www.haskell.org/mailman/listinfo/web-devel
>>
>
>
>
>
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel