[Haskell-cafe] Optimizing a high-traffic network architecture

[Joel Reymont]

On Dec 15, 2005, at 2:02 PM, Simon Marlow wrote:

>> The statistics are phys/VM, CPU usage in % and #packets/transfer  
>> speed
>>
>> Total:   1345, Lobby:   1326, Failed:  0, 102/184, 50%, 90/8kb
>> Total:   1395, Lobby:   1367, Failed:  2
>> Total:   1421, Lobby:   1394, Failed:  4
>> Total:   1490, Lobby:   1463, Failed:  4, 108/194, 50%, 110/11Kb
>> Total:   1574, Lobby:   1546, Failed:  4, 113/202, 50%, 116/11kb
>
> Hmm, your machine is spending 50% of its time doing nothing, and the
> network traffic is very low.  I wouldn't expect 2k connections to pose
> any problem at all, so further investigation is definitely required.

That's CPU utilization by the program. My laptop is actually running  
a lot of other stuff as well, although the other stuff is not  
consuming much CPU.

> With 2k connections the overhead of select() is going to start to be a
> problem.  You would notice the system time going up.  -threaded may  
> help
> with this, because it calls select() less often.

I'm testing 4k connections now but I think the app is spending most  
of the time collecting garbage :-). Well, running handlers on those  
keep-alive packets as well to update internal state.

I think I would need to profile next. I would love to see a report of  
data in drag/void state but it's impossible since I'm using STM.  
Unless I can hack support for STM into profiling myself (unlikely?  
any pointers?) I think I'll have to move away from STM just to  
profile the program.

	Joel


--
http://wagerlabs.com/