[web-devel] Performance of Yesod 0.7 fourteen times slower than Rails 2.3?

[Sven Koschnicke (GFXpro)]

Hello,

I tested again with persistent-postgresql 0.4.0.1 and now the results 
are what you would expect :) rails in dev mode is now 79 times 
slower[1]. Of course, thats not a fair compare, but Greg did some 
testing with rails production already. Thanks for improving that so quickly!

Greetings
   Sven

[1] https://github.com/SKoschnicke/performance-test

On 03/25/2011 08:41 AM, Michael Snoyman wrote:
> Hi Sven,
>
> Can I ask you to try things out one more time, this time with
> persistent-postgresql 0.4.0.1? I think I fixed some more of the
> issue[1].
>
> Thanks,
> Michael
>
> [1] http://docs.yesodweb.com/blog/improving-persistent-performance
>
> On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)
> <s.koschnicke at gfxpro.com>  wrote:
>> Great that you found a way to speed things up! I'm looking forward to the
>> blog post. I will run some more tests with the improved code, just let me
>> know when its available. Is someone working on improving the postgres
>> database backend? Maybe I could help there, the bad performance with
>> postgres is really a showstopper for me.
>>
>> Sven
>>
>> On 03/23/2011 08:13 AM, Michael Snoyman wrote:
>>>
>>> This was a great example of a performance bug being exactly where you
>>> don't expect it. I'm planning on writing up a blog post on this
>>> adventure later, but for the quick summary: the Criterion benchmarks
>>> have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200
>>> fold speedup for some operations. This came from switching a large
>>> monad transformer stack to a single RWS monad. But more on that in the
>>> blog post.
>>>
>>> I've run your test a few more times: it still seems like there's a
>>> slowdown from the database code, but I think the widget performance
>>> bug was the big issue. After some more testing on my end, I'll put the
>>> code on Hackage. Would you give the test another shot after that?
>>>
>>> Thanks again for bringing this up!
>>>
>>> Michael
>>>
>>> On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)
>>> <s.koschnicke at gfxpro.com>    wrote:
>>>>
>>>> Hello,
>>>>
>>>> I think your guess wasn't completely wrong. I tested with Sqlite now, and
>>>> with that rails and yesod are equally slow (rails 11.81 secs vs yesod
>>>> 11.36
>>>> secs). But its also interesting that your code change speeds it up.
>>>>
>>>> (I put the sqlite benchmark at
>>>> https://github.com/SKoschnicke/performance-test/tree/sqlite)
>>>>
>>>> My first guess was actually that its the marshalling which slows yesod so
>>>> much and that rails uses some clever optimizations there, but I haven't
>>>> got
>>>> enough insight to validate that.
>>>>
>>>> Greetings
>>>>   Sven
>>>>
>>>>
>>>>
>>>> On 03/21/2011 01:49 PM, Michael Snoyman wrote:
>>>>>
>>>>> Looks like my initial guess was completely wrong: on my system, almost
>>>>> no time is spent on the database query. Instead, it looks like there's
>>>>> some kind of performance issue with Hamlet. For example, changing line
>>>>> 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases
>>>>> performance dramatically (25 sec to 4.5 sec). I'll need to spend some
>>>>> more time to properly diagnose the problem. Again, thanks for bringing
>>>>> this to my attention.
>>>>>
>>>>> Michael
>>>>>
>>>>> On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman<michael at snoyman.com>
>>>>>   wrote:
>>>>>>
>>>>>> Firstly, thank you for running this benchmark: it's the only way we
>>>>>> can find out where Yesod needs to be fixed.
>>>>>>
>>>>>> I haven't had a chance to look into the code properly, but it looks
>>>>>> like this is more a test of the database backend than of the web
>>>>>> framework. I'll try to get to this myself later, but it would be very
>>>>>> interesting to see the difference in response times between PostgreSQL
>>>>>> and sqlite. Currently, persistent-sqlite binds directly to the sqlite
>>>>>> C API, while persistent-postgresql uses HDBC. It's entirely possible
>>>>>> that HDBC is adding some overhead.
>>>>>>
>>>>>> One of the items on our wish list[1] is to migrate away from HDBC.
>>>>>> Having hard numbers like you are providing will help us figure out our
>>>>>> priorities a bit better.
>>>>>>
>>>>>> Thanks,
>>>>>> Michael
>>>>>>
>>>>>> [1] http://wiki.yesodweb.com/Wishlist
>>>>>>
>>>>>> On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)
>>>>>> <s.koschnicke at gfxpro.com>      wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I'm coming from a Rails background and was very impressed by the
>>>>>>> features of
>>>>>>> the Yesod Web Framework. Working on an trading app in Rails I wanted
>>>>>>> to
>>>>>>> port
>>>>>>> some features of this app to a Yesod app just to learn the framework
>>>>>>> and
>>>>>>> because we have some performance issues with Rails.
>>>>>>>
>>>>>>> The first thing I noticed was that the Yesod app was significantly
>>>>>>> slower
>>>>>>> than the Rails app at trivial tasks. I am very astonished about that.
>>>>>>> I
>>>>>>> thought Rails should be slower because of its interpreted nature. I
>>>>>>> made
>>>>>>> some benchmarking and measured that a simple Rails app is fourteen
>>>>>>> times
>>>>>>> faster than a Yesod app that did the same thing (loading some records
>>>>>>> from
>>>>>>> the database and displaying them). Did I made a mistake or is my
>>>>>>> understanding, that the compiled app should be faster, just wrong?
>>>>>>>
>>>>>>> I documented the results on github:
>>>>>>>
>>>>>>> https://github.com/SKoschnicke/performance-test
>>>>>>>
>>>>>>> Greetings
>>>>>>>   Sven Koschnicke
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> web-devel mailing list
>>>>>>> web-devel at haskell.org
>>>>>>> http://www.haskell.org/mailman/listinfo/web-devel
>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>