[Haskell-beginners] Network client - reading and writing to a socket

[Chaddaï Fouché]

On Mon, Aug 1, 2011 at 9:16 PM, Manfred Lotz <manfred.lotz at arcor.de> wrote:
> On Sun, 31 Jul 2011 15:14:23 -0400
> Patrick LeBoutillier <patrick.leboutillier at gmail.com> wrote:
>
>> Manfred,
>>
>> > The problem is that the message itself is some 30K big and I only
>> > get some 16K of the message.
>> >
>> > How could I force to get the whole message?
>>
>> My guess is that you can't. This call:
>>
>> c' <- B.hGetNonBlocking h 40000
>>
>> tries to read as much as it can (up to 40000 bytes) but it won't block
>> to wait for data. Perhaps the rest of your message is in a different
>> TCP packet or delayed or whatever, but I think you have to keep on
>> reading (and maybe block) until you know you have read the entire
>> message. The IMAP specs will tell you how to identify the "end of the
>> message".
>>
>> BTW: This issue is not Haskell specific. If you implement the same
>> code in C, Perl or Java you will have to deal with the same problem.
>> When you read from a socket, there is no general way of knowing that
>> the other side has sent everything.
>>
>
> Hmm. I'm not quite sure you are fully right. On the one hand I believe
> that this could be an issue which arises in python/perl etc. as well.
> On the other hand I believe it should be possible to receive from a
> socket what is available at a certain point of time.

Yes and that's exactly what you have done : your hGetLine blocks until
there is something to read on the socket and then your hGetNonBlocking
gets *everything* there is to read on this socket at this exact
moment... Except as had been said by others that your message has been
split into several packets and they're not all there when you hGet.

What you want is not "what is available at a certain point of time",
you want to read a whole message, except that this "message" notion is
only in your head (and in a specific protocol, IMAP here) it has no
direct relevance to how things happens on the network.

> I found this link http://sequence.complete.org/node/257, and when I run
> the code I get the full message from the imap server even if the
> message is a couple of megabytes big.

Of course you get the whole message ! This code try to read (with
blocking calls) forever what's on your socket, it reads it, send it on
a TChan and then retry reading it, bit by bit it gets your whole
message, of course it has no idea that it got your whole message and
if nothing is done it will continue to wait on your socket for all
eternity...
The key point is this function :

> listenLoop :: IO a -> TChan a -> IO ()
> listenLoop act chan =
>       sequence_ (repeat (act >>= atomically . writeTChan chan))

This does not stop short of an exception. Two of those loops are
started each in their own thread (so that they don't block the rest of
the program) to read stdin and a socket respectively.

> I have to figure out how to use the code for my need as I do not get
> the input from the keyboard.

This code is probably not doing what you want, this is a toy example
where most of the complexity comes from handling two source of input
simultaneously and collating their answer on the same TChan. Its main
problem is that it don't even try to read messages, it just read
everything and doesn't know when it's finished. You need to listen to
David's advice and read a bit on the protocol you're trying to handle
and how it encode the notion of message, how it signals that the
message has ended.

-- 
Jedaï