[Haskell-cafe] Is Haskell a Good Choice for Web Applications? (ANN: Vocabulink)

[Jason Dagit]

On Wed, May 6, 2009 at 3:54 PM, Anton van Straaten
<anton at appsolutions.com> wrote:
> FWIW, I have an internal HAppS application that's been running continuously
> since November last year, used daily, with stable memory usage.

Do you have advice about the way you wrote you app?  Things you
knowingly did to avoid space leaks?  Maybe a blog about your HAppS
app?

>> I have also experienced it with
>> writing a forever loop in Haskell that did polling from channels.  I
>> would leave my app running for, say, 4 hours and it would be using
>> tons of memory.
>
> If you posted an example of that, there are probably people who'd be
> interested in debugging it.

I'm not sure if I still have the source code or not.  I did ask people
at the time and the only thing I recall now was that I was told to
make sure that I fully evaluate the thunks I create on each iteration
of my forever loop.  This was at least 2 years ago now.  I don't know
if I ever resolved it as it was just a toy benchmark anyway and I
wasn't really happy with the results.

>> I think it's fair to say that keeping the memory usage low of a long
>> running Haskell app is hard, but that is a general issue not just a
>> Haskell issue.  It's hard in most languages.
>
> I don't agree with this.  This should not be hard in language
> implementations with good garbage collectors, that don't have known
> limitations (this excludes e.g. conservative GC and reference-counting
> systems).  In my experience, it's not hard to write stable long-running code
> in good implementations of languages like Haskell, Scheme, Common Lisp, or
> Java.

There are certainly cases where no automatic garbage collector could
know when it is safe to collect certain things.  A quick google search
for java space leaks turned up this article:
http://www.ibm.com/developerworks/java/library/j-leaks/

I think wikipedia uses the term "logical leak" for the type of space
leak I'm thinking of.  The garbage collector thinks you care about an
object but in fact, you want it to be freed.  Yes, it's because of a
bug, but these are bugs that tend to be subtle and tedious.

>> I think what we need to
>> address this is more information about preventative measures.  What
>> programming styles cause the problem and which ones solve it.  I would
>> say that I lack confidence recommending anyone to use Haskell for long
>> running processes because I don't understand well the problem of
>> keeping the usage low.  If it is a well documented problem with
>> documented solutions (more than just profiling), then I would regain
>> my confidence because I know the problem can be worked around
>> reliably.  Does this make sense?  Maybe it's already well documented?
>
> This doesn't completely make sense to me, in that either a program has a
> space leak, or it doesn't.  If it does, you debug it and resolve it.  If a
> leak really is due to a problem with the language implementation or standard
> libraries, then that should be identified and reported.  There shouldn't be
> any deep mystery here.  At the very least, it should be possible to point to
> the relevant trac tickets if there really is a problem.

The ambiguity is me thinking of relative cost of finding/fixing these
bugs.  Testing for correctness is something we tend to automate very
well.  See unit testing for example.  But, testing for space leaks is
not something we have automated in my experience.  Take Don's comment
that you let it run for a while and then look at the profiling graph.
How do you automate this and make it part of your automatic test
suite?  It seems like something that requires manual testing.  So
maybe you do some QA prior to releases?  That sounds reasonable to me.
 The darcs team collects misbehaving repositories and has tools to
automate the collection of statistics so that at regular intervals
they can check the performance of darcs to make sure it's getting
better or staying the same with each release (in practice I think
these stats are collected by buildbots on each applied patch bundle).
So then, at some point we must have a bag of tricks for dealing with
these space leaks.  I want to talk about those tricks.  I'm not
talking about bugs in a specific program, but instead about techniques
and styles that are known to work well in practice.

I hope that clarifies things a bit.

Thanks,
Jason