[Haskell-cafe] Network.HTTP+ByteStrings Interface--Or: How to shepherd handles and go with the flow at the same time?

[Thomas Schilling]

On 5/24/07, haskell at list.mightyreason.com <haskell at list.mightyreason.com> wrote:

> Why do you need to convert Socket to Handle?

Initially, we chose to use socketToHandle for simplicity reasons--why
duplicate functionality if we can reuse it?  After Simon Marlow's
comment that my reason to assume it inappropriate does no longer hold,
we decided to just keep it that way.  I'm not completely sure, but I
think that handle created from the socket will not use up any of the
file handles, or if it does then it's the socket behind it, but in any
case it doesn't change anything w.r.t. to the total available number
of resource handles.

If it turns out that we should in fact add functions to work directly
on sockets then it would be mostly straightforward translation of
hGet, hGetNonBlocking and hGetContents.  But that's not an issue, ATM.

> If this hypothetical 'socketGetContentsAsLazyByteString' also creating a
> newEmptyMVar then it could return a (tryPutMVar m ()) action to allow the
> consumer of the lazy string to signal to the deferred reading process (which
> periodically calls (isEmptyMVar m)) that it should close the socket or at least
> stop trying to read from it.  That way there is no _need_ to 'seq' your way to
> the end of the lazy bytestring to cause it to close.

Now here comes the actual problem, sorry if that wasn't clear from my
original post.  We actually don't want to rely on the consumer to
signal that it is done reading--this would be manual memory management
and this is unreliable.  Also, we can't be sure the string will ever
be consumed completely.  Another unsatisfactory solution is to attach
a finalizer which is executed when the garbage collector detects that
the object is not live anymore, which is not guaranteed at all.

These are precisely the reasons why Oleg proposed the fold-based
method.  Unfortunately, this does not seem to work nicely in its
current form.

Since, I want to keep some form of laziness though, my trade-off is to
create a "lazy zone" in which we read data on demand, but if someone
needs to hold on to it longer he has to gather the data explicitly,
which, as showed, might be tricky to use, thus undesirable (especially
for a library).

/ Thomas

-- 
"Remember! Everytime you say 'Web 2.0' God kills a startup!" -
userfriendly.org, Jul 31, 2006