[web-devel] Performance of Yesod 0.7 fourteen times slower than Rails 2.3?

[Greg Weber]

using RAILS_ENV=production ./script/server

ree 1.8.7 ruby-pg driver
21s
ruby 1.9 pg driver
16-17s

given there are a few time columns I tried "gem install home_run"
and RAILS_ENV=production home_run ./script/server
It didn't seem to make a difference- I think time object creation is lazy in
Rails.

Yesod with persistent-postgresql-0.4.0.1 gets 17s

But then I looked at the code.
For the template it is using: addWidget $(widgetFile "quote-index")
I changed it too: addHamlet $(hamletFile "quote-index")

and the time dropped down to 13-14s

So finally, we are beating Rails- if we can't beat Rails on a benchmark that
should be considered a horrible defect :)

Here is a patch to fix widgetFile, although we should probably investigate
why addWidget is slower.
https://github.com/gregwebs/yesod/commit/32d11707f3d5bec73a5c3002f3924c58d82112a6

On Fri, Mar 25, 2011 at 12:41 AM, Michael Snoyman <michael at snoyman.com>wrote:

> Hi Sven,
>
> Can I ask you to try things out one more time, this time with
> persistent-postgresql 0.4.0.1? I think I fixed some more of the
> issue[1].
>
> Thanks,
> Michael
>
> [1] http://docs.yesodweb.com/blog/improving-persistent-performance
>
> On Wed, Mar 23, 2011 at 9:23 AM, Sven Koschnicke (GFXpro)
> <s.koschnicke at gfxpro.com> wrote:
> > Great that you found a way to speed things up! I'm looking forward to the
> > blog post. I will run some more tests with the improved code, just let me
> > know when its available. Is someone working on improving the postgres
> > database backend? Maybe I could help there, the bad performance with
> > postgres is really a showstopper for me.
> >
> > Sven
> >
> > On 03/23/2011 08:13 AM, Michael Snoyman wrote:
> >>
> >> This was a great example of a performance bug being exactly where you
> >> don't expect it. I'm planning on writing up a blog post on this
> >> adventure later, but for the quick summary: the Criterion benchmarks
> >> have gone from 23.7ms (== 23,700us) to 141us. Meaning this is a 200
> >> fold speedup for some operations. This came from switching a large
> >> monad transformer stack to a single RWS monad. But more on that in the
> >> blog post.
> >>
> >> I've run your test a few more times: it still seems like there's a
> >> slowdown from the database code, but I think the widget performance
> >> bug was the big issue. After some more testing on my end, I'll put the
> >> code on Hackage. Would you give the test another shot after that?
> >>
> >> Thanks again for bringing this up!
> >>
> >> Michael
> >>
> >> On Mon, Mar 21, 2011 at 5:41 PM, Sven Koschnicke (GFXpro)
> >> <s.koschnicke at gfxpro.com>  wrote:
> >>>
> >>> Hello,
> >>>
> >>> I think your guess wasn't completely wrong. I tested with Sqlite now,
> and
> >>> with that rails and yesod are equally slow (rails 11.81 secs vs yesod
> >>> 11.36
> >>> secs). But its also interesting that your code change speeds it up.
> >>>
> >>> (I put the sqlite benchmark at
> >>> https://github.com/SKoschnicke/performance-test/tree/sqlite)
> >>>
> >>> My first guess was actually that its the marshalling which slows yesod
> so
> >>> much and that rails uses some clever optimizations there, but I haven't
> >>> got
> >>> enough insight to validate that.
> >>>
> >>> Greetings
> >>>  Sven
> >>>
> >>>
> >>>
> >>> On 03/21/2011 01:49 PM, Michael Snoyman wrote:
> >>>>
> >>>> Looks like my initial guess was completely wrong: on my system, almost
> >>>> no time is spent on the database query. Instead, it looks like there's
> >>>> some kind of performance issue with Hamlet. For example, changing line
> >>>> 20 of Handler/Quote.hs from "addWidget" to "addHtml" increases
> >>>> performance dramatically (25 sec to 4.5 sec). I'll need to spend some
> >>>> more time to properly diagnose the problem. Again, thanks for bringing
> >>>> this to my attention.
> >>>>
> >>>> Michael
> >>>>
> >>>> On Mon, Mar 21, 2011 at 12:40 PM, Michael Snoyman<michael at snoyman.com
> >
> >>>>  wrote:
> >>>>>
> >>>>> Firstly, thank you for running this benchmark: it's the only way we
> >>>>> can find out where Yesod needs to be fixed.
> >>>>>
> >>>>> I haven't had a chance to look into the code properly, but it looks
> >>>>> like this is more a test of the database backend than of the web
> >>>>> framework. I'll try to get to this myself later, but it would be very
> >>>>> interesting to see the difference in response times between
> PostgreSQL
> >>>>> and sqlite. Currently, persistent-sqlite binds directly to the sqlite
> >>>>> C API, while persistent-postgresql uses HDBC. It's entirely possible
> >>>>> that HDBC is adding some overhead.
> >>>>>
> >>>>> One of the items on our wish list[1] is to migrate away from HDBC.
> >>>>> Having hard numbers like you are providing will help us figure out
> our
> >>>>> priorities a bit better.
> >>>>>
> >>>>> Thanks,
> >>>>> Michael
> >>>>>
> >>>>> [1] http://wiki.yesodweb.com/Wishlist
> >>>>>
> >>>>> On Mon, Mar 21, 2011 at 12:10 PM, Sven Koschnicke (GFXpro)
> >>>>> <s.koschnicke at gfxpro.com>    wrote:
> >>>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I'm coming from a Rails background and was very impressed by the
> >>>>>> features of
> >>>>>> the Yesod Web Framework. Working on an trading app in Rails I wanted
> >>>>>> to
> >>>>>> port
> >>>>>> some features of this app to a Yesod app just to learn the framework
> >>>>>> and
> >>>>>> because we have some performance issues with Rails.
> >>>>>>
> >>>>>> The first thing I noticed was that the Yesod app was significantly
> >>>>>> slower
> >>>>>> than the Rails app at trivial tasks. I am very astonished about
> that.
> >>>>>> I
> >>>>>> thought Rails should be slower because of its interpreted nature. I
> >>>>>> made
> >>>>>> some benchmarking and measured that a simple Rails app is fourteen
> >>>>>> times
> >>>>>> faster than a Yesod app that did the same thing (loading some
> records
> >>>>>> from
> >>>>>> the database and displaying them). Did I made a mistake or is my
> >>>>>> understanding, that the compiled app should be faster, just wrong?
> >>>>>>
> >>>>>> I documented the results on github:
> >>>>>>
> >>>>>> https://github.com/SKoschnicke/performance-test
> >>>>>>
> >>>>>> Greetings
> >>>>>>  Sven Koschnicke
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> web-devel mailing list
> >>>>>> web-devel at haskell.org
> >>>>>> http://www.haskell.org/mailman/listinfo/web-devel
> >>>>>>
> >>>>>
> >>>
> >>>
> >
> >
>
> _______________________________________________
> web-devel mailing list
> web-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/web-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/web-devel/attachments/20110326/c4775c2a/attachment-0001.htm>