[Haskell-cafe] Memory consumption issues under heavy network throughput/concurrency loads

[Gregory Collins]

On Wed, Jul 16, 2014 at 5:46 AM, Ben Bangert <ben at groovie.org> wrote:

> I've run the new server code, indeed it initially takes less memory.
> However, when I ctrl-C the testing client, it fails to properly close
> sockets now, they stay open forever (the original code I posted always
> closed the sockets when I killed the testing client). Did you try ctrl-c
> the test client and re-running it several times?
>

Yes once I exhibited the leaking behaviour I stopped (and didn't implement
cleanup properly). It should be ok now.


I'm guessing that the recv call is blocking forever and failing somehow to
> note that the socket was actually closed under it, while the hGetline I was
> using properly detects this condition and closes the socket when it notices
> the other side has left.
>

The Handle stuff will still end up calling recv() under the hood.


I should note that when asking for even 1000 echo clients, the test client
> as you've changed it dies when I try to launch it (it shows Clients
> Connected: 0, then just exits silently).
>

Are you running out of FDs?



> I had to use my old testing client to test the server changes you made. If
> the sockets would properly close when terminating the client abruptly, then
> its quite possible memory usage would remain lower as it definitely took
> much much less memory for the first 2k echo clients.
>

I've also updated my copy of the the test code to work with the GHC 7.4.1
that's installed on my Ubuntu box at work (64-bit) --- and I *don't* see
the client leak there, resident heap plateaus at around 90MB.

G
-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20140716/d70b7f11/attachment-0001.html>