[web-devel] Kick-off discussion: Yesod 0.9

[Michael Snoyman]

On Wed, Jun 1, 2011 at 9:10 AM, Blake Rain
<blake.rain at massivetactical.com> wrote:
> Hey guys,
>
> Some random thoughts while I have some time :)
>
>
> On Mon, 2011-05-30 at 23:41 +0300, Michael Snoyman wrote:
>> Hi all,
>>
>> It's been about a month since the 0.8 release of Yesod. In the release
>> announcement[1], we laid out some goals. Let's take a chance to
>> analyze where we're at:
>>
>> * Documentation: Still improving as always. In particular, the new
>> Yesod site is making it much easier for me to write new content (I
>> have a few things on my local system that can finally go up).
>
> Excellent :)
> I also very much like the wiki. Perhaps a cook-book section?

Sounds good. Personally, I'm going to be putting my examples up as
blog posts, but I strongly encourage people to add their own examples
as well, and if someone wants to start a world-editable topic linking
those examples together, I'd be happy to give it a prominent link.

>> * Static file caching headers: Implemented by Greg Weber. We'll likely
>> be making some API changes to handle embedded files, though that
>> discussion deserves its own blog post.
>
> This would be extremely useful: We currently serve static files
> from a different domain using a different server (NGinX I think). If
> Warp was to handle this out of the box then a measure of complexity
> would be removed from our server configuration. Which is always of
> benefit.

Even though Warp will be able to handle this, there would likely be
two reasons to stick with your current setup:

1) Nginx on a separate domain won't involve cookie overhead.
2) Nginx will likely be a bit faster than Warp for static files.

The first issue, however, could be mitigated by using virtual hosting
within your Warp setup. And I'm not sure how significant the
performance difference is between nginx and warp.

> Aside: I do need to have a chat with you (Michael et al) at some point
> about deployment strategies and WAI applications when we (Massive) get
> onto the next two projects (December mostly likely, unless I find some
> more staff).

I'm feeling much more confident about this after the deployment of the
new Yesod site. Setting up EC2 couldn't have been simpler: start an
instance, install nginx and postgresql, configure upstart for nginx
and the Yesod app, and copy the executable. I think this process could
easily be turned into a script, and customized for different
persistent backends.

>> * Other template types: For Hamlet 0.9, we're going to be splitting
>> Julius/Coffeescript into their own package, and juggling the type
>> signature for Coffeescript a bit. At that point, it should be
>> straightforward to embed Coffeescript directly into a Yesod app, just
>> like Julius is today.
>
> Is CoffeeScript really any better than JavaScript? Are any Haskellers
> having any success using it? Would the CoffeeScript compiler still need
> to be installed on the server to be invoked by the Yesod application? Or
> would it be done at compile time?

The CoffeeScript compiler needs to be available on the server at
runtime. I personally don't use it at all.

[snip]

>> * Forms package: It's been rewritten, but hasn't really had much input
>> from others. Now's the time to speak up if you have any opinions on
>> it. The code is available on Hackage.
>
> I am hoping to take a look at the new forms package shortly. I had to
> write all our form processing manually due to various complexities that
> could not be covered by yesod-forms. Would a description of these be
> useful?

Yes, incredibly useful. Currently, yesod-forms is scratching all my
itches, so I need to know what is missing to improve it.

>> * Embedded objects in MongoDB: No work done yet.
>
> Would this not make persistent more complex? Is it necessary to do so?

I'm hoping not. The goal is to keep persistent the same.

> We migrated away from persistent and PostgreSQL to MongoDB on a
> suggestion by Greg Weber (we felt acid-state is not yet mature enough
> for our purposes). We used an in-house abstraction on top of the MongoDB
> Haskell driver to allow easier marshalling, object embedding,
> persistent-like keys and such.
>
> Unfortunately, this did mean we had to fight a lot with some
> deficiencies we found in Data.BSON. Also we had to refactor almost the
> entire site, which was quite good fun :)
>
>>
>> * Performance improvements: Nothing in particular. If you have any
>> examples of Yesod performing badly, please let us know.
>
> I've got examples of it performing extremely well ;)

Those are good too.

>>
>> The goal is for Yesod 0.9 to be feature complete, and for 1.0 to be a
>> fairly stable API. From the list above, my two priorities are better
>> Javascript loading (require.js) and static file serving. However,
>> there's one other major issue we need to address: the devel server.
>> Unfortunately, it doesn't work very well at all right now. Thankfully,
>> "yesod build" *does* work very well, and for my development I've
>> fallen back to simply "yesod build && ./dist/build/myapp/myapp" on the
>> command line. My guess is that we're 90% of the way there on "yesod
>> devel". This is a project that isn't tightly bound to the rest of the
>> API work, and would be very approachable for someone trying to get
>> started on Yesod development. If you're interested in helping out
>> here, please let me know.
>
> It is a bit broken. Our current (and first) project using Yesod has 85
> modules, 93 hamlet files, etc... The devel server just takes too long
> and sometimes dies. It also consumes an inordinate amount of memory (2-3
> Gb).

I'm guessing this was with "wai-handler-devel", right? yesod 0.8
introduced a *new* broken devel server, "yesod devel" :). But its
problem shouldn't be memory usage. If so, I'd like to hear more.

> At the moment I use GHCi, a typical session being:
>
> 1. Make some changes
> 2. `touch` .hs file if Hamlet file is modified
> 3. :reload
> 4. main
> 5. Test in browser (or whatever)
> 6. Ctrl+C
> 7. goto 1
>
> I've been working this way for some time now, and it actually seems to
> be working very well. Our designer gets upset -- but he always does :P
>
> Only thing I might do is reduce `settingsTimeout` in the argument to
> `runSettings` in Warp to something more useful for debugging (like 5).
>
>>
>> I have no specific timeframe in mind for Yesod 0.9. As it stands now,
>> it looks like the 0.9 release will involve almost no API breakage,
>> which is a very good sign. But if you have any ideas you'd like to
>> contribute, I'd recommend getting them in in the next week or two.
>
>
> Ultimately, I'd prefer Yesod was correct, rather than on time :)
>
> In spite of any issues Yesod has, We are using it in a production
> environment very successfully. Our client and their investor(s) are very
> satisfied with the project, and it will be released to the public
> shortly.
>
> This weekend I tested scalability, and ran the beta application on a
> cluster to test load-balancing, fail-over and so on (mostly whilst
> getting used to MongoDB). It performed beautifully.
>
> If anything, I'd prefer work was concentrated on the forms package to
> make it magically cater for all my needs ;) But I suppose I should
> define those needs first.
>
> Again, many thanks for all your hard work!
>
> - B.
>
>
>>
>> Michael
>>
>> [1] http://www.yesodweb.com/blog/2011/4/announcing-yesod-0-8
>>
>> _______________________________________________
>> web-devel mailing list
>> web-devel at haskell.org
>> http://www.haskell.org/mailman/listinfo/web-devel
>
>
>