[web-devel] Performance of Yesod 0.7 fourteen times slower than Rails 2.3?

[Michael Snoyman]

I would definitely appreciate some help with the PostgreSQL backend.
Here's the basic game plan I have in mind right now:

* Split out the TH code into a separate package. This will be done for
a 0.5 release.
* Review Aur's join code. Most likely that will get included immediately.
* Switch over the PostgreSQL backend from HDBC to the PostgreSQL C API.

It's the last step that I would really appreciate help with. It looks
like we might use the libpq package[1]. The only downside I see is
that it has a unix package dependency, meaning it won't work on
Windows. I see a few possibilities:

* Don't worry about it. Windows users can't use PostgreSQL. (Bad option IMO.)
* Get libpq to convert over to unix-compat if possible.
* Maintain both the HDBC backend and a libpq backend.

Frankly, given how Persistent is structured internally, it shouldn't
be very complicated to make a migration to a different backend. If
you'd like to take a crack at it, email me off-list about how to get
started (or just fork the repo on Github and get going).

Thanks,
Michael

[1] http://hackage.haskell.org/package/libpq

On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)
<s.koschnicke at gfxpro.com> wrote:
> Great that you found a way to speed things up! I'm looking forward to the
> blog post. I will run some more tests with the improved code, just let me
> know when its available. Is someone working on improving the postgres
> database backend? Maybe I could help there, the bad performance with
> postgres is really a showstopper for me.
>
> Sven
>
> On 03/23/2011 08:13 AM, Michael Snoyman wrote:
>>
>> This was a great example of a performance bug being exactly where you
>> don't expect it. I'm planning on writing up a blog post on this
>> adventure later, but for the quick summary: the Criterion benchmarks
>> have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200
>> fold speedup for some operations. This came from switching a large
>> monad transformer stack to a single RWS monad. But more on that in the
>> blog post.
>>
>> I've run your test a few more times: it still seems like there's a
>> slowdown from the database code, but I think the widget performance
>> bug was the big issue. After some more testing on my end, I'll put the
>> code on Hackage. Would you give the test another shot after that?
>>
>> Thanks again for bringing this up!
>>
>> Michael
>>
>> On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)
>> <s.koschnicke at gfxpro.com>  wrote:
>>>
>>> Hello,
>>>
>>> I think your guess wasn't completely wrong. I tested with Sqlite now, and
>>> with that rails and yesod are equally slow (rails 11.81 secs vs yesod
>>> 11.36
>>> secs). But its also interesting that your code change speeds it up.
>>>
>>> (I put the sqlite benchmark at
>>> https://github.com/SKoschnicke/performance-test/tree/sqlite)
>>>
>>> My first guess was actually that its the marshalling which slows yesod so
>>> much and that rails uses some clever optimizations there, but I haven't
>>> got
>>> enough insight to validate that.
>>>
>>> Greetings
>>>  Sven
>>>
>>>
>>>
>>> On 03/21/2011 01:49 PM, Michael Snoyman wrote:
>>>>
>>>> Looks like my initial guess was completely wrong: on my system, almost
>>>> no time is spent on the database query. Instead, it looks like there's
>>>> some kind of performance issue with Hamlet. For example, changing line
>>>> 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases
>>>> performance dramatically (25 sec to 4.5 sec). I'll need to spend some
>>>> more time to properly diagnose the problem. Again, thanks for bringing
>>>> this to my attention.
>>>>
>>>> Michael
>>>>
>>>> On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman<michael at snoyman.com>
>>>>  wrote:
>>>>>
>>>>> Firstly, thank you for running this benchmark: it's the only way we
>>>>> can find out where Yesod needs to be fixed.
>>>>>
>>>>> I haven't had a chance to look into the code properly, but it looks
>>>>> like this is more a test of the database backend than of the web
>>>>> framework. I'll try to get to this myself later, but it would be very
>>>>> interesting to see the difference in response times between PostgreSQL
>>>>> and sqlite. Currently, persistent-sqlite binds directly to the sqlite
>>>>> C API, while persistent-postgresql uses HDBC. It's entirely possible
>>>>> that HDBC is adding some overhead.
>>>>>
>>>>> One of the items on our wish list[1] is to migrate away from HDBC.
>>>>> Having hard numbers like you are providing will help us figure out our
>>>>> priorities a bit better.
>>>>>
>>>>> Thanks,
>>>>> Michael
>>>>>
>>>>> [1] http://wiki.yesodweb.com/Wishlist
>>>>>
>>>>> On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)
>>>>> <s.koschnicke at gfxpro.com>    wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm coming from a Rails background and was very impressed by the
>>>>>> features of
>>>>>> the Yesod Web Framework. Working on an trading app in Rails I wanted
>>>>>> to
>>>>>> port
>>>>>> some features of this app to a Yesod app just to learn the framework
>>>>>> and
>>>>>> because we have some performance issues with Rails.
>>>>>>
>>>>>> The first thing I noticed was that the Yesod app was significantly
>>>>>> slower
>>>>>> than the Rails app at trivial tasks. I am very astonished about that.
>>>>>> I
>>>>>> thought Rails should be slower because of its interpreted nature. I
>>>>>> made
>>>>>> some benchmarking and measured that a simple Rails app is fourteen
>>>>>> times
>>>>>> faster than a Yesod app that did the same thing (loading some records
>>>>>> from
>>>>>> the database and displaying them). Did I made a mistake or is my
>>>>>> understanding, that the compiled app should be faster, just wrong?
>>>>>>
>>>>>> I documented the results on github:
>>>>>>
>>>>>> https://github.com/SKoschnicke/performance-test
>>>>>>
>>>>>> Greetings
>>>>>>  Sven Koschnicke
>>>>>>
>>>>>> _______________________________________________
>>>>>> web-devel mailing list
>>>>>> web-devel at haskell.org
>>>>>> http://www.haskell.org/mailman/listinfo/web-devel
>>>>>>
>>>>>
>>>
>>>
>
>