[Haskell-cafe] "too many open files" using snap

[Ozgun Ataman]

Not knowing how exactly you set up your interop with mysql, is there any chance your application tries to keep too many mysql connections open? A simple "show processlist;" inside of a mysql console should rule out this possibility.

Also, are you reading to or writing from any files at any point independently of the snap infrastructure? I have had issues in the past with intermittent background file reads not being strict enough and eventually starving the system of file descriptors.

Cheers,
Ozgun



On Sunday, January 8, 2012 at 4:27 PM, Gregory Collins wrote:

> On Sun, Jan 8, 2012 at 7:25 AM, Eric Wong <wsysdu at gmail.com (mailto:wsysdu at gmail.com)> wrote:
> > Hi haskellers,
> > 
> > I wrote a web server using snap framework. Basically, it's an API server using hdbc-mysql to connect to a mysql server. But it also serves some static web pages and images, using the serveDirectory function in snap-core. The program eats more and more memory slowly, but I couldn't spot the leak. It uses less than 10MB memory when starting. Sometimes, it can grow to 60-70MB and stop working correctly. Then I find that the log file is full of "too many open files" error. As I don't have any code open any files, I suspect it's the serveDirectory leaking. But I don't have any expertise on the core of snap, so, could some one give me some advice on this problem?  
> 
> I'm fairly certain there's no leak in Snap in this code, although I will gladly accept evidence to the contrary -- I've been load-testing "serveDirectory" at 20,000 qps for a couple of minutes now, with no increase in the memory resident size.
> 
> A "too many open files" error is usually due to running out of file descriptors for network sockets. On my Linux machine the per-process limit for file descriptors defaults to 1024, and on my Mac the default is 256. You can see your current limit by running "ulimit -a" from the terminal.
> 
> Each incoming socket occupies a file descriptor, as does every open connection to your database and every open file. When these run out, calls to accept() or to open files start failing and you get this error message. 
> 
> Memory profiling should help you find the leak.
> 
> G
> -- Gregory Collins <greg at gregorycollins.net (mailto:greg at gregorycollins.net)>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org (mailto:Haskell-Cafe at haskell.org)
> http://www.haskell.org/mailman/listinfo/haskell-cafe
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120108/99145447/attachment.htm>