[Haskell-cafe] ANN: conduit-network-stream, A base layer for network protocols with Conduits

[Alexander V Vershilov]

Thanks for explanation.

However I'm still confused as if I'll use one of protocols (protobuf, json,
sereal message, binary message) all of them has
iteration interface i.e. function of type (ByteString -> Result a) that
returns
data Result = Error Text ByteString | NeedMore (ByteString -> Result a) |
HaveResult a
all of them can be used in conduits using sinkParser or same approach and
so there will be
no problem in sending and receiving messages even if they are split, and
that will be done in a logical block.

I see that there will be problems if I'll use raw data and I see that your
library solves it, am I right?



On 25 February 2013 15:53, Nils <mail at nils.cc> wrote:

> Hi Alexander,
>
> Am 25.02.2013 06:52, schrieb Alexander V Vershilov:
>
>  Can you describe if there any difference with latest conduit API
>> (yield, await) that can be
>> used to write functions in a very similar style, but without using
>> exeternal packages:
>>
>
> I have indeed written this library for the 0.5.6 API, so things might have
> changed a bit in the 1.0 API, but I'd be surprised if the fundamental flaws
> of this approach would have been fixed.
>
>
>   >  runTCPServer settings app = appSource ap $$ go =$= CL.mapM_ encode
>> =$ appSink app
>>  >     where
>>  >         go = forever $ do
>>  >            bp <- decode <$> await
>>
>
> `await` works on a conduit of strict `ByteString` chunks. The size/length
> of each `ByteString` is dependent on your network connection, so if you
> have a fast internet connection and your bytestrings are sufficiently
> separated because you're waiting for a response between each message your
> program might actually work as you expect it to do (with a little bit of
> luck). But consider a simple server/client application where the messages
> are not seperated by a small delay:
>
>     server = runTCPServer (..) $ \ad ->
>         appSource ad $$ Data.Conduit.List.mapM_ (liftIO . print)
>     client = runTCPClient (..) $ \ad ->
>         (yield "msg1" >> yield "msg2") $$ appSink ad
>
> The server will simply print out "msg1msg2" as one message, not as two
> separate messages. Even worse, if your network connection is bad or your
> chunks are getting too big for buffering, you might end up with something
> like:
>
>     "msg1ms"
>     "g2"
>
> `await` is not reliable in that regard because the network source is not
> consistent and non-deterministic. My libraray makes sure that every "yield"
> from the client corresponds to exactly one (not more or less) "await" at
> the server.
>
> There are more benefits when using my library. For example consider a
> client which first sends an authorization message, then a couple of hashes
> from different files and then maybe some timestamps on some other files.
> Writing that server is straight forward:
>
>     client = runTCPClient (..) $ \ad ->
>         send1    ad $$ yield authenticationMsg
>         sendList ad $$ mapM_ yield [file1hash, file2hash, file3hash]
>         sendList ad $$ mapM_ yield [timestamp1, timestamp2]
>
>     server = runTCPServer (..) $ \ad ->
>         (next,[authmsg])  <- receive ad   $$ Data.Conduit.List.consume
>         (next,hashes)     <- receive next $$ Data.Conduit.List.consume
>         (next,timestamps) <- receive next $$ Data.Conduit.List.consume
>         close next
>
> Each `receive` corresponds to exactly one `send`. Without this library you
> have to manually check/verify/associate each message by hand in one big
> loop, whereas this library allows you to split your application into
> logical conduit "groups" which are straight forward to use.
>
>
> - Nils
>



-- 
Alexander
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20130225/839b2f50/attachment.htm>