[Haskell-cafe] network package and SIGVTALRM

[David Turner]

Hi Ruben,

I think you're falling into a common trap re. TCP. On a lightly-loaded
network, if you send a block of data on one host it typically arrives at
the other end of the connection as one thing. In other words, calls to
send() and recv() are one-to-one. In that situation adding NODELAY will
(seem to) solve problems like the ones that you were seeing. However, it
will all fall to pieces when you're running under load or there's
congestion or some other kind of problem, as it's perfectly legitimate for
packets to be combined and/or fragmented which breaks this one-to-one
relationship on which the correctness of your program rests.

You _must_ treat data received over TCP as a continuous stream of bytes and
not a sequence of discrete packets, and do things such as accounting for
the case where your 4-byte length indicator is split across two packets so
does not all arrive at once. If you don't, it will bite you at the very
worst time, and will do so nondeterministically. This kind of thing is very
hard to reproduce in a test environment.

There is nothing special about the DCC protocol that makes it immune from
this effect.

Best wishes,

David





On 14 July 2016 at 10:48, Ruben Astudillo <ruben.astud at gmail.com> wrote:

> On 13/07/16 14:54, Andrey Sverdlichenko wrote:
>
>> If you are lucky. They still may be merged by sender if retransmission
>> occurs, or on receiving side, if receiver waits too long before reads, and
>> this is up to OS scheduler to control.
>> NDELAY option is used to improve interactive latency, it will not make TCP
>> obey message boundaries.
>>
>
> You're right. But understanding a little better the problem maybe will
> clarify why NODELAY is a valid option. DCC is in parts a redundant
> protocol. When you connect, the senders gives you the file you want but
> for coherency reasons every once in a while you have to reply the current
> transfered size through the same socket. This was a mean of preserving
> consistency that is redundant by the same mechanisms implemented on TCP.
> From the page[1] I am using to implement
>
>   ``client A sends blocks of data (usually 1-2 KB) and at every block
> awaits
>   confirmation from the client B, that when receiving a block should reply
>   4 bytes containing an positive number specifying the total size of the
>   file received up to that moment.
>
>   The transmission closes when the last acknowledge is received by client
> A.
>
>   The acknowledges were meant to include some sort of coherency check in
>   the transmission, but in fact no client can recover from an acknowledge
>   error/desync, all of them just close the connection declaring the
>   transfer as failed (the situation is even worse in fact, often
>   acknowledge errors aren't even detected!).
>
>   Since the packet-acknowledge round trip eats a lot of time, many clients
>   included the send-ahead feature; the client A does not wait for the
>   acknowledge of the first packet before sending the second one.''
>
> The last part explains why my download still succeded until half the size
> of the file. But no sending any reply (because the message is too little
> to send) is a failure of interactivity on the protocol, not of message
> boundaries (which specify that the reply is 4 byte in length).
>
> [1]: http://www.kvirc.net/doc/doc_dcc_connection.html
> --
> -- Ruben Astudillo
>
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20160714/2afa9cc4/attachment.html>