[web-devel] On the state of Haskell web frameworks

[Gregory Collins]

Michael Snoyman <michael at snoyman.com> writes:

> [F]irstly, I believe your premise is mistaken: we're not trying to
> "standardize interfaces" for the most part; it's true that WAI was
> such an approach, but that's not the focus of this thread.

If it's not the focus, it's a recurring theme, especially in the context
of multiple active efforts in that direction and postings on the list
like this:

> The Haskell language itself was created to consolidate fragmented
> market of different FP languages. Are we ready to follow our
> 'ancestors' and in their spirit do the same when it comes to
> web-framework(s)?

OK, obviously we're all on the same "team" here, we're all drinking the
same kool-aid and there should be plenty of things we can agree upon:
Haskell's not tremendously popular, there are only so many decent
Haskell hackers, it would be good for us to pool our efforts and work
towards common purposes. And there IS a lot of interest in and support
of the idea of standardizing a web application interface for haskell:
I've seen lots of positive chatter and off the top of my head I can
think of at least three on Hackage, yours, Hack, and CGI.

CGI has a historical reason to exist (it understands the CGI protocol)
but the other two are "just plumbing". Which is fine!  Plumbing is
great, life would be really horrible without it. But in our case, I
don't think it's necessary right now: we don't have good toilets or
sewers! The marginal utility of producing a WAI/Hack adapter for our
toolkit at this point in time is basically zero: we're talking about
spending time making it easier to interoperate with *applications that
don't exist*. In fact, the only ones I can even think of at the moment
are gitit, and MAYBE the Hackage server. Chicken/egg, right?


> you claim that anyone could write an adapter between two interfaces;
> look at the code for hack-handler-hyena and you'll understand the
> folly of that sentiment.

Are you kidding? It's 164 lines of not-that-hairy code, a full third of
which is a status code lookup table. (OK, I'm going to try to be less
argumentative :) )


> But more to the point: because of WAI, we don't *have* to write the
> adapter,

This is the "M*N" VS "M+N" argument that the Rack guys make, which would
be a perfectly good argument except that for us (pretty much)
M=N=2. Keep in mind that WSGI (and to a lesser extent, this applies to
rack) was created in the context of a bunch of webserver backends and
more than half a dozen huge Python web toolkits, any one of which
probably had more person-hours invested in it than all of our stuff put
together.


>  and I don't see any substantive critiques of the WAI approach. (If
> there are, we can address them by fixing the WAI.)

...and I just said I was going to try to be less argumentative.

Your iteratee formulation is wrong, for starters. You picked the one the
current Hyena is using, which Johan is currently in the process of
abandoning because the other way is clearly better -- an explicit
existential variable for the state is not as clean as the other
formulation which pushes that goop down under the lambda, into a
continuation. The Iteratee type from the iteratee library also has a
convenient Monad instance (a big win!). I don't see that one is possible
here, especially since there's no way that I can see for your iteratee
to partially consume an input; how can you compose them?

The Source thing is a little wacky, it should be an Enumerator
instead. Again, the point of these things is that they *compose*;
iteratees, which consume chunks of data, plug into enumerators, which
produce them. So the request body should be an enumerator, and you
process it by providing it an iteratee to read the data; the response
body is an enumerator the user fills out, which plugs into an iteratee
that splats it out the socket.

Speaking of which, you don't define an iteratee type, given your
formulation it should be:

    type Iteratee a = a -> ByteString -> IO (Either a a)

or more generally

    type IterateeM m a = a -> ByteString -> m (Either a a)

which is exactly the Hyena definition.

The Status datatype is a mess. The requestHeader field should probably
be a Map instead of an alist. RequestHeader and ResponseHeader don't
need to have all of those header constructors in the datatype like that;
as a result you have "ReqContentLength" which is awkward to say the
least. Not to mention the fact that web application code should never
have to get its hands dirty dealing with content-lengths for requests
anyways; that should be handled on the enumerator level.

Let's say I use the generic "RequestHeader ByteString" constructor. HTTP
header names are supposed to be case-insensitive, and your Eq instance
does a case-sensitive comparison: 'RequestHeader "x-foo" /=
RequestHeader "X-Foo"', which is wrong; now I can't use "lookup" to pick
a header out of the alist.

Request has an entry for remoteHost but not remotePort. Request contains
no cookies, query parameters, or POST parameters; I'm supposed to parse
them myself? Users of Java servlets are snickering.


I'm really not trying to badger you here (honest), I'm being hyperbolic
to make a point: for nearly all of the criticisms I just made, there's
room for a legitimate disagreement about what is the "best" way to
approach the solution; lots of people wouldn't want to use iteratees at
all, for instance! I also wouldn't expect you to rearrange the whole
thing to cater to my whims or anyone else's, either.

I think that if we're going to standardize, one of the following
preconditions has to hold:

 * the standard is superior on engineering merits, solves concrete
   problems/enables you to get work done, and is agreeable enough to
   enough people that it picks up community momentum by wide
   consensus.

 * the standard is imposed by fiat, maybe because it comes from the
   language vendor.

 * the standard comes out of a standardization committee effort which
   hammers out disagreements over a period of time to produce something
   which everyone can stomach --- I'd actually be pretty okay with this
   if it happened, but there's a reason "design by committee" is a
   perjorative.

 * the standard comes about as a result of market forces, i.e. blub
   toolkit becomes really popular and a standard emerges because
   everyone wants to interoperate with blub.

Honestly, I just don't any of these things happening, not right now.


> Now, let me try to understand your point here: we need standardized
> interfaces for applications to communicate with each other, but we
> can't do that until we have the low level interfaces hammered out.

No, you're misrepresenting me here. What I said is that "we need
pluggable applications that understand how talk to each other", NOT that
those interfaces should be subject to standardization. There's a
critical distinction there; you don't expect Django components to plug
into a Zope app.


> On the other hand, you say that it's premature to try to standardize
> interfaces. And then you make it clear that the use case of many
> people (shared hosting) doesn't interest you at all; based on previous
> conversations, I believe this means that Snap will have *no* (Fast)CGI
> support.

I wouldn't say that's necessarily true in the long term, although it's
probably true that I myself am not likely to write it. FastCGI support
is definitely not important to me, but if someone wanted it badly enough
there's nothing in Snap (as of this moment) which would preclude her
from writing whichever backend she wanted: CGI, WAI, Hack,
whatever. That said, I'd be 100% a-ok with sacrificing CGI support
specifically, because spawning a fresh process every time you want to
service a request is, frankly, dumb and absurdly limiting. It isn't 1996
anymore.


> I think (correct me if I'm wrong) that everyone else here is in
> agreement that the goal should not be to create a great framework that
> will solve everyone's needs.

I do agree with this; my point all long has been that no one web library
is going to satisfy everyone any more than one parsing combinator
library or one regular expression library or one database could. I'm
aiming to help create a great framework that satisfies my needs, and if
it works for other people too: great.


> The goal should be to create great libraries that can be used
> independently.

..but not necessarily this, at least not in the broader context of
"people writing web applications". Obviously I agree in general terms:
self-contained, single-purpose libraries are great. There's no reason to
have a bunch of competing libraries to parse and output RSS, for
instance.

A productive *web framework*, on the other hand, by its nature makes a
whole set of "convention over configuration" assumptions about how
requests are handled and routed, how data is accessed, how users are
managed, how to hook into an admin panel, how templates are laid out on
disk, which template system to ship out of the box, how components are
specified, how they talk to each other etc etc etc. I see the future
being a lot like the present, where a heterogeneous collection of
frameworks of varying comprehensiveness and quality provide their own
self-contained ecosystems.

G.
-- 
Gregory Collins <greg at gregorycollins.net>