[Haskell-cafe] ANN: conduit-network-stream, A base layer for network protocols with Conduits

[Nils]

Hi Alexander,

Am 25.02.2013 06:52, schrieb Alexander V Vershilov:
> Can you describe if there any difference with latest conduit API
> (yield, await) that can be
> used to write functions in a very similar style, but without using
> exeternal packages:

I have indeed written this library for the 0.5.6 API, so things might 
have changed a bit in the 1.0 API, but I'd be surprised if the 
fundamental flaws of this approach would have been fixed.

>  >  runTCPServer settings app = appSource ap $$ go =$= CL.mapM_ encode
> =$ appSink app
>  >     where
>  >         go = forever $ do
>  >            bp <- decode <$> await

`await` works on a conduit of strict `ByteString` chunks. The 
size/length of each `ByteString` is dependent on your network 
connection, so if you have a fast internet connection and your 
bytestrings are sufficiently separated because you're waiting for a 
response between each message your program might actually work as you 
expect it to do (with a little bit of luck). But consider a simple 
server/client application where the messages are not seperated by a 
small delay:

     server = runTCPServer (..) $ \ad ->
         appSource ad $$ Data.Conduit.List.mapM_ (liftIO . print)
     client = runTCPClient (..) $ \ad ->
         (yield "msg1" >> yield "msg2") $$ appSink ad

The server will simply print out "msg1msg2" as one message, not as two 
separate messages. Even worse, if your network connection is bad or your 
chunks are getting too big for buffering, you might end up with 
something like:

     "msg1ms"
     "g2"

`await` is not reliable in that regard because the network source is not 
consistent and non-deterministic. My libraray makes sure that every 
"yield" from the client corresponds to exactly one (not more or less) 
"await" at the server.

There are more benefits when using my library. For example consider a 
client which first sends an authorization message, then a couple of 
hashes from different files and then maybe some timestamps on some other 
files. Writing that server is straight forward:

     client = runTCPClient (..) $ \ad ->
         send1    ad $$ yield authenticationMsg
         sendList ad $$ mapM_ yield [file1hash, file2hash, file3hash]
         sendList ad $$ mapM_ yield [timestamp1, timestamp2]

     server = runTCPServer (..) $ \ad ->
         (next,[authmsg])  <- receive ad   $$ Data.Conduit.List.consume
         (next,hashes)     <- receive next $$ Data.Conduit.List.consume
         (next,timestamps) <- receive next $$ Data.Conduit.List.consume
         close next

Each `receive` corresponds to exactly one `send`. Without this library 
you have to manually check/verify/associate each message by hand in one 
big loop, whereas this library allows you to split your application into 
logical conduit "groups" which are straight forward to use.


- Nils