[Haskell-cafe] Memory consumption issues under heavy network throughput/concurrency loads

[Ben Bangert]

On Jul 15, 2014, at 1:49 PM, Gregory Collins <greg at gregorycollins.net> wrote:

> I took your test case and hacked it down to eliminate some possible sources of error (I was suspicious of Handle for a while, also of an old space leak bug in "forever" which I think is fixed now):
> 
>     https://github.com/gregorycollins/ghc-echo-leak-bug
> 
> The revised echoserver is fine on my machine (stable at 22MB resident) but the echo client leaks. Happens with/without -O2 on GHC 7.8.3 for OSX.

I've run the new server code, indeed it initially takes less memory. However, when I ctrl-C the testing client, it fails to properly close sockets now, they stay open forever (the original code I posted always closed the sockets when I killed the testing client). Did you try ctrl-c the test client and re-running it several times?

I'm guessing that the recv call is blocking forever and failing somehow to note that the socket was actually closed under it, while the hGetline I was using properly detects this condition and closes the socket when it notices the other side has left.

I should note that when asking for even 1000 echo clients, the test client as you've changed it dies when I try to launch it (it shows Clients Connected: 0, then just exits silently). I had to use my old testing client to test the server changes you made. If the sockets would properly close when terminating the client abruptly, then its quite possible memory usage would remain lower as it definitely took much much less memory for the first 2k echo clients.

Cheers,
Ben


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20140715/d0e0ff81/attachment.sig>