[Haskell-cafe] Haskell vs. Erlang for heavy-duty network apps (was Re: Haskell Speed)

[Joel Reymont]

On Dec 25, 2005, at 10:13 AM, Bulat Ziganshin wrote:

> Hello Joel,
> [...]
> so i think that your problems is due to bad design decisions caused by
> lack of experience. two weeks ago when you skipped my suggestions
> about improving this design and answered that you will use "systematic
> approach", i foresee that you will fail and say that Haskell is a bad
> language

Yes and no. The systematic approach that I used was profiling the  
serialization code and tweaking all that I could. I saved my  
profiling reports after each run and tracked the changes that I made.  
I will blog about it after Simon M. comes back and suggests how to  
squeeze the last bit out of it.

Regardless of this, it looks to me like I could easily have around  
4Mb of network traffic per second with about 4k threads and  
complicated nested structures to serialize and deserialize. Trying to  
tackle far less data suggests to me that it's not gonna happen. So I  
will try to take this as far as I can in Haskell, once I have the  
heavy artillery to back me up. If the results are good then I will  
use them in later applications of the same nature but in the meantime  
I'm rewriting this particular app in Erlang.

There are other apps to write in Haskell but my overall lesson is  
that the choice of right tool for the task at hand will help far more  
than any optimizations you can produce. Specially when the natural  
approach to the problem will just produce the results required.

I spent 3 months with this app, going from total Haskell newbie who  
has not seen Haskell before to someone who can read Haskell core code  
and tackle FFI and networking in a heavily concurrent environment. I  
still don't know what "the right thing" is.

The Erlang rewrite should take me 2-3 days with FFI being the teeth- 
grinding bit. I need the FFI (which totally sucks compared to  
Haskell's) because my use of SSL and ZLib is non-standard, otherwise  
these are already provided to you. I expect that the app will just  
work. No memory or time leaks. The serialization approach that I'm  
using is based on the pickler combinators but slightly different. I  
think I'll backport it to Haskell to compare.

The next project on my plate is real-time collusion detection in  
poker. I first thought of using the Dazzle approach and Bayesian  
networks but the focus again seems to be on high-throughput  
networking and scalability. What language will I use? Haskell is no  
sure bet.

A disclaimer for those who might think that I'm a newbit  
disillutioned with Haskell... I totally love Haskell, it just seems  
that you need to be a guru to do what I'm trying to do and I have not  
yet reached this status. I will not rest until I'm able to see just  
_what_ kind of a performance can be squeezed from Haskell for my  
application. I'll also continue to seek enlightenment but I need to  
finish this project before New Year's.

	Joel

--
http://wagerlabs.com/