[Haskell-cafe] sendfile leaking descriptors on Linux?
tphyahoo at gmail.com
Sun Feb 28 03:14:52 EST 2010
As far as I can tell, stepcut was probably correct in his diagnosis before.
I was getting two processes started because of a stray cron job.
2010/2/26 Thomas Hartman <tphyahoo at gmail.com>:
> Indeed, the error occurs when two processes are running at the same
> time. One process isn't serving anything, and the other is just
> looping, reported that it can't stop because the port is taken, and
> repeating ad infinitum.
> The loop I have is
> ulimit -v 150000
> while true; do
> echo starting patchtag: `date`
> ./dist/build/patchtagserver/patchtagserver --port=80 --
> echo patchtag exited: `date`
> echo patchtag exited: `date` >> patch-tag.log
> rm -f _local/patchtagserver_state.lock
> The ulimit is so that it will die after consuming a reasonable amount
> of memory. (I seem to have a memory leak somewhere, but it takes many
> hours to get up to 150M.)
> Everything else looks pretty standard to me. It seems to work as
> designed most of the time, but occasionally will result in two
> processes. I'm baffled as to how this happens.
> If anybody has any idea, I would love to hear them.
> 2010/2/26 Jeremy Shaw <jeremy at n-heptane.com>:
>> It will be interesting to see if that makes any difference -- it shouldn't.
>> In happstack-server we use 'listenOn'. According to the documentation
>> listenOn already sets ReuseAddr:
>> A quick look at the source code confirms it is calling:
>> setSocketOption sock ReuseAddr 1
>> It sounds to be like you are getting the socket in use error because you
>> have 2 processes running. Seems like the first one hasn't really died, but
>> the second one is started anyway. How is it that your loop starts a second
>> server when the first one has not finished?
>> - jeremy
>> On Fri, Feb 26, 2010 at 1:43 PM, Thomas Hartman <tphyahoo at gmail.com> wrote:
>>> Thanks, I altered my top level request handler as follows
>>> mysmartserver conf h stateProxy = do
>>> socket <- bindPort conf
>>> <I added (setSocketOption socket ReuseAddr 1) here>
>>> <Should this be added in a comment, or even in function code,
>>> in Happstack.Server.SimpleHTTP? What are the tradeoffs, when would you
>>> *not* want ot use ReuseAddr?)
>>> webserverTid <- forkIO $ simpleHTTPWithSocket socket conf h
>>> putStrLn . ( "starting happs server" ++ ) =<< time
>>> control <- startSystemState stateProxy -- start the HAppS state
>>> putStrLn . ( "happs state started" ++ ) =<< time
>>> killThread webserverTid
>>> stateShutdown control
>>> I can't replicate the error reliably so I won't know if it actually
>>> fixed the problem for a while, therefore just leaving this cookie
>>> crumb trail in case others may find it helpful.
>>> 2010/2/26 Brandon S. Allbery KF8NH <allbery at ece.cmu.edu>:
>>> > On Feb 26, 2010, at 04:28 , Thomas Hartman wrote:
>>> >> me: Like mightybyte, I run my app in a shell loop that will just
>>> >> restart it after a crash. But every once in a while it won't restart
>>> >> because of the busy socket and I need to do a manual restart, killing
>>> >> multiple processes (usually 2).
>>> >> the error I get is:
>>> >> bind: resource busy (Address already in use)
>>> > This is on application restart? It's not out of file descriptors, it's
>>> > just
>>> > the system keeping the socket around (netstat will show it in TIME_WAIT,
>>> > or
>>> > possibly in a shutdown negotiation state such as LAST_ACK, FIN_WAIT,
>>> > etc.
>>> > TCP lacks *reliable* socket shutdown negotiation).
>>> > You want to configure the socket with SO_REUSEADDR before trying to bind
>>> > it
>>> > (setSocketOption socket ReuseAddr 1).
>>> > As to the portable version of sendfile, it's because not all systems
>>> > offer a
>>> > sendfile() system call. Linux and *BSD do, and can use the native
>>> > implementation; the portable version emulates sendfile() when it doesn't
>>> > exist, at the price of additional CPU usage/system load (sendfile()
>>> > having
>>> > been created specifically to reduce system load in the common case for
>>> > web
>>> > servers).
>>> > --
>>> > brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery at kf8nh.com
>>> > system administrator [openafs,heimdal,too many hats] allbery at ece.cmu.edu
>>> > electrical and computer engineering, carnegie mellon university KF8NH
More information about the Haskell-Cafe