[web-devel] On the state of Haskell web frameworks

[Michael Snoyman]

On Mon, Mar 15, 2010 at 10:04 PM, Gregory Collins
<greg at gregorycollins.net>wrote:

> Michael Snoyman <michael at snoyman.com> writes:
>
> > [F]irstly, I believe your premise is mistaken: we're not trying to
> > "standardize interfaces" for the most part; it's true that WAI was
> > such an approach, but that's not the focus of this thread.
>
> If it's not the focus, it's a recurring theme, especially in the context
> of multiple active efforts in that direction and postings on the list
> like this:
>
> > The Haskell language itself was created to consolidate fragmented
> > market of different FP languages. Are we ready to follow our
> > 'ancestors' and in their spirit do the same when it comes to
> > web-framework(s)?
>
> OK, obviously we're all on the same "team" here, we're all drinking the
> same kool-aid and there should be plenty of things we can agree upon:
> Haskell's not tremendously popular, there are only so many decent
> Haskell hackers, it would be good for us to pool our efforts and work
> towards common purposes. And there IS a lot of interest in and support
> of the idea of standardizing a web application interface for haskell:
> I've seen lots of positive chatter and off the top of my head I can
> think of at least three on Hackage, yours, Hack, and CGI.
>
> CGI has a historical reason to exist (it understands the CGI protocol)
> but the other two are "just plumbing". Which is fine!  Plumbing is
> great, life would be really horrible without it. But in our case, I
> don't think it's necessary right now: we don't have good toilets or
> sewers! The marginal utility of producing a WAI/Hack adapter for our
> toolkit at this point in time is basically zero: we're talking about
> spending time making it easier to interoperate with *applications that
> don't exist*. In fact, the only ones I can even think of at the moment
> are gitit, and MAYBE the Hackage server. Chicken/egg, right?
>
> I don't think it's really fair to put CGI in the same category, and the
Hack/WAI issue is well documented: Hack is simple, WAI is performant.
However, let me address the meat of your claim: even though you don't see
the value, doesn't mean there is none. I'm able to write an application
against the WAI and test it locally with the SimpleServer, then deploy it
with CGI. That by itself justifies its existence for me.

>
> > you claim that anyone could write an adapter between two interfaces;
> > look at the code for hack-handler-hyena and you'll understand the
> > folly of that sentiment.
>
> Are you kidding? It's 164 lines of not-that-hairy code, a full third of
> which is a status code lookup table. (OK, I'm going to try to be less
> argumentative :) )
>
> It wasn't the line count I was talking about: it requires spawning a
separate thread to convert the Enumerator to a lazy bytestring.

>
> > But more to the point: because of WAI, we don't *have* to write the
> > adapter,
>
> This is the "M*N" VS "M+N" argument that the Rack guys make, which would
> be a perfectly good argument except that for us (pretty much)
> M=N=2. Keep in mind that WSGI (and to a lesser extent, this applies to
> rack) was created in the context of a bunch of webserver backends and
> more than half a dozen huge Python web toolkits, any one of which
> probably had more person-hours invested in it than all of our stuff put
> together.
>
>
> I don't really see your point here.


> >  and I don't see any substantive critiques of the WAI approach. (If
> > there are, we can address them by fixing the WAI.)
>
> ...and I just said I was going to try to be less argumentative.
>
> Your iteratee formulation is wrong, for starters. You picked the one the
> current Hyena is using, which Johan is currently in the process of
> abandoning because the other way is clearly better -- an explicit
> existential variable for the state is not as clean as the other
> formulation which pushes that goop down under the lambda, into a
> continuation. The Iteratee type from the iteratee library also has a
> convenient Monad instance (a big win!). I don't see that one is possible
> here, especially since there's no way that I can see for your iteratee
> to partially consume an input; how can you compose them?
>
> The Source thing is a little wacky, it should be an Enumerator
> instead. Again, the point of these things is that they *compose*;
> iteratees, which consume chunks of data, plug into enumerators, which
> produce them. So the request body should be an enumerator, and you
> process it by providing it an iteratee to read the data; the response
> body is an enumerator the user fills out, which plugs into an iteratee
> that splats it out the socket.
>
> So, instead of speaking rhetoric, can you give a concrete example where the
Source is *bad*? It has serious benefits over Enumerator, such as the
ability to convert (via unsafeInterleaveIO) into a lazy bytestring.

I'm happy to see a better approach, why not give me a datatype for the
"obviously better" Enumerator?

(Hint: releasing no code and using phrases like "clearly better" are not
going to win any arguments.)


> Speaking of which, you don't define an iteratee type, given your
> formulation it should be:
>
>    type Iteratee a = a -> ByteString -> IO (Either a a)
>
> or more generally
>
>    type IterateeM m a = a -> ByteString -> m (Either a a)
>
> which is exactly the Hyena definition.
>

The Status datatype is a mess. The requestHeader field should probably
> be a Map instead of an alist. RequestHeader and ResponseHeader don't
> need to have all of those header constructors in the datatype like that;
> as a result you have "ReqContentLength" which is awkward to say the
> least. Not to mention the fact that web application code should never
> have to get its hands dirty dealing with content-lengths for requests
> anyways; that should be handled on the enumerator level.
>
> That just shows a misunderstanding on your part: the content-lengths are
*not* handled at the application level, though the information is available
to the application. Why would I censor it? And would you care to elaborate
on "Status datatype is a mess"?


> Let's say I use the generic "RequestHeader ByteString" constructor. HTTP
> header names are supposed to be case-insensitive, and your Eq instance
> does a case-sensitive comparison: 'RequestHeader "x-foo" /=
> RequestHeader "X-Foo"', which is wrong; now I can't use "lookup" to pick
> a header out of the alist.
>
> This is a valid concern. However, my goal in that decision is to keep
things as close to the wire as possible. On the wire, x-foo and X-Foo are
different. You also are conveniently ignoring all character encoding issues.
True, HTTP has a character encoding defined, but do you trust ever web
client in the world to respect all this?


> Request has an entry for remoteHost but not remotePort. Request contains
> no cookies, query parameters, or POST parameters; I'm supposed to parse
> them myself? Users of Java servlets are snickering.
>
> Yes, you are. You clearly have a misunderstanding about what WAI is about
(or are just trying to attack baselessly): it's meant to represent the raw
data.

(And if anyone wants remotePort for any reason, let me know. I have yet to
see a need for it, and no one has asked.)

>
> I'm really not trying to badger you here (honest), I'm being hyperbolic
> to make a point: for nearly all of the criticisms I just made, there's
> room for a legitimate disagreement about what is the "best" way to
> approach the solution; lots of people wouldn't want to use iteratees at
> all, for instance! I also wouldn't expect you to rearrange the whole
> thing to cater to my whims or anyone else's, either.
>
> I think that if we're going to standardize, one of the following
> preconditions has to hold:
>
>  * the standard is superior on engineering merits, solves concrete
>   problems/enables you to get work done, and is agreeable enough to
>   enough people that it picks up community momentum by wide
>   consensus.
>
>  * the standard is imposed by fiat, maybe because it comes from the
>   language vendor.
>
>  * the standard comes out of a standardization committee effort which
>   hammers out disagreements over a period of time to produce something
>   which everyone can stomach --- I'd actually be pretty okay with this
>   if it happened, but there's a reason "design by committee" is a
>   perjorative.
>
>  * the standard comes about as a result of market forces, i.e. blub
>   toolkit becomes really popular and a standard emerges because
>   everyone wants to interoperate with blub.
>
> Honestly, I just don't any of these things happening, not right now.
>
>
> Don't worry, I won't ask you to join the standardization committee ;).


> > Now, let me try to understand your point here: we need standardized
> > interfaces for applications to communicate with each other, but we
> > can't do that until we have the low level interfaces hammered out.
>
> No, you're misrepresenting me here. What I said is that "we need
> pluggable applications that understand how talk to each other", NOT that
> those interfaces should be subject to standardization. There's a
> critical distinction there; you don't expect Django components to plug
> into a Zope app.
>
>
> > On the other hand, you say that it's premature to try to standardize
> > interfaces. And then you make it clear that the use case of many
> > people (shared hosting) doesn't interest you at all; based on previous
> > conversations, I believe this means that Snap will have *no* (Fast)CGI
> > support.
>
> I wouldn't say that's necessarily true in the long term, although it's
> probably true that I myself am not likely to write it. FastCGI support
> is definitely not important to me, but if someone wanted it badly enough
> there's nothing in Snap (as of this moment) which would preclude her
> from writing whichever backend she wanted: CGI, WAI, Hack,
> whatever. That said, I'd be 100% a-ok with sacrificing CGI support
> specifically, because spawning a fresh process every time you want to
> service a request is, frankly, dumb and absurdly limiting. It isn't 1996
> anymore.
>
>
> > I think (correct me if I'm wrong) that everyone else here is in
> > agreement that the goal should not be to create a great framework that
> > will solve everyone's needs.
>
> I do agree with this; my point all long has been that no one web library
> is going to satisfy everyone any more than one parsing combinator
> library or one regular expression library or one database could. I'm
> aiming to help create a great framework that satisfies my needs, and if
> it works for other people too: great.
>
>
> > The goal should be to create great libraries that can be used
> > independently.
>
> ..but not necessarily this, at least not in the broader context of
> "people writing web applications". Obviously I agree in general terms:
> self-contained, single-purpose libraries are great. There's no reason to
> have a bunch of competing libraries to parse and output RSS, for
> instance.
>
> A productive *web framework*, on the other hand, by its nature makes a
> whole set of "convention over configuration" assumptions about how
> requests are handled and routed, how data is accessed, how users are
> managed, how to hook into an admin panel, how templates are laid out on
> disk, which template system to ship out of the box, how components are
> specified, how they talk to each other etc etc etc. I see the future
> being a lot like the present, where a heterogeneous collection of
> frameworks of varying comprehensiveness and quality provide their own
> self-contained ecosystems.
>
>
Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/web-devel/attachments/20100316/e0f2126d/attachment-0001.html