[Haskell-cafe] Network.Curl cookie jar madness

[Iustin Pop]

On Sun, Aug 19, 2012 at 06:06:53PM +0200, Iustin Pop wrote:
> On Sun, Aug 19, 2012 at 12:45:47AM -0400, Michael Orlitzky wrote:
> > On 08/18/2012 08:52 PM, Michael Orlitzky wrote:
> > > I'm one bug away from a working program and need some help. I wrote a
> > > little utility that logs into LWN.net, retrieves an article, and creates
> > > an epub out of it.
> > 
> > I've created two pages where anyone can test this. The first just takes
> > any username and password via post and sets a session variable. The
> > second prints "Success." if the session variable is set, and "Failure."
> > if it isn't. The bash script,
> 
> […]
> 
> > The attached haskell program using Network.Curl, doesn't:
> > 
> >   $ runghc haskell-test.hs
> >   Logged in...
> >   Failure.
> > 
> > Any help is appreciated =)
> 
> So, take this with a grain of salt: I've been bitten by curl (the
> haskell bindings, I mean) before, and I don't hold the quality of the
> library in great regard.
> 
> The libcurl documentation says: "When you set a file name with
> CURLOPT_COOKIEJAR, that file name will be created and all received
> cookies will be stored in it when curl_easy_cleanup(3) is called" (i.e.
> at the end of a curl handle session). But even though the curl bindings
> seem to run easy_cleanup on handles (initialize → mkCurl →
> mkCurlWithCleanup), they don't do this correctly:
> 
> DEBUG: ALLOC: CURL
> DEBUG: ALLOC: /tmp/network-curl-test-haskell20417.txt
> DEBUG: ALLOC: username=foo&password=bar
> DEBUG: ALLOC: http://michael.orlitzky.com/tmp/network-curl-test1.php
> DEBUG: ALLOC: WRITER
> DEBUG: ALLOC: WRITER
> 
> Note there's no "DEBUG: FREE: CURL" as the code seems to imply there
> should be. Hence, the handle is never cleaned up (do the curl bindings
> leak handles?), so the cookie file is never written.
> 
> Side note: by running the same program multiple times, sometimes you see
> DEBUG: FREE: CURL, sometimes no FREE actions. I believe there's
> something very wrong in the curl bindings with regard to cleanups.

On more investigation, this seems to be due to the somewhat careless use
of Foreign.Concurrent; from the docs:

  “The finalizer will be executed after the last reference to the
  foreign object is dropped. There is no guarantee of promptness, and in
  fact there is no guarantee that the finalizer will eventually run at
  all.”

Also, see http://hackage.haskell.org/trac/ghc/ticket/1364.

So it seems that the intended way of cleaning up curl handles is all
fine and dandy if one doesn't require timely cleanup; in most cases,
this is not needed, but for cookies it is broken.

I don't know what the proper solution is; either way, it seems that
there should be a way to force the cleanup to be run, via
finalizeForeignPtr, or requiring full manual handling of curl handles
(instead of via finalizers).

Gah, native libs++.

regards,
iustin