[Haskell-cafe] Long-running request/response protocol server using enumerator/iterator/iterIO/pipes/conduits/...

[Paolo Capriotti]

On Tue, Jun 26, 2012 at 8:22 PM, Nicolas Trangez <nicolas at incubaid.com> wrote:
> Hello Cafe,
>
> Some time ago I tried to implement a network service using iteratee (or
> enumerator, can't remember), but gave up in the end. More recently I
> wanted to create something similar (a similar protocol), but failed
> again.
>
> So I'm looking for some example code or something similar (Google only
> helped slightly).
>
> First of all, I don't care which API/library to use, I guess for my
> purpose all of enumerator, iteratee, iterIO, pipes, conduits,... are OK,
> so all feedback is welcome.
>
> Here's the catch. Most examples out there implement some server which
> accepts a single client request, interprets it, creates a response,
> returns this, and closes the connection (or something alike, think
> HTTP).
>
> The protocol I'd like to implement is different: it's long-running using
> repeated requests & responses on a single client connection. Basically,
> a client connects and sends some data to the server (where the length of
> this data is encoded in the header). Now the server reads & parses this
> (binary) data, sets up some initial state for this client connection
> (e.g. opening a file handle), and returns a reply. Now the client can
> send another request, server parses/interprets it using the connection
> state, sends a reply, and so on.
>
> Might sound easy (and actually it's pretty easy in most other languages
> I know, including an OCaml implementation), yet I fail to figure out how
> to get this done using some enumerator-style library.

With the current development version of pipes-core
(https://github.com/pcapriotti/pipes-core/tree/devel) I would write something
like the following (completely untested) code:

    import qualified Control.Pipe.Binary as B
    ...

    request :: PipeL IO ByteString ByteString u ()
    request = do
        h <- header
        let n = hdrSize h
        B.take n

    -- I assume a fixed-size header for simplicity
    header :: PipeL IO ByteString b u Header
    header = do
        h <- B.take headerSize >+> fold (<>) ByteString.empty
        return $ parseHeader h -- the function doing the actual parsing

    handler :: Pipe IO ByteString ByteString u ()
    handler = do
        -- server logic here
        -- just echo the input data as an example
        void idP

    server hInput hOutput
           -- read from the socket
         = B.handleReader hInput
           -- process all requests
       >+> forever (withUnawait $ request >+> handler)
           -- write to socket
       >+> B.handleWriter hOutput

Requests are handled sequentially by the `forever` loop. This whole pipeline
works with chunks of data represented as `ByteString`s, and `PipeL` (a new
feature of pipes-core 0.2.0) is used to pass leftover data along.

In a real implementation, you would also probably need to wrap `handler` in
something like:

    catch handler $ \e ->
        liftIO $ logException e
    discard

so that failures (or early termination) in `handler` don't bring the whole
pipeline down.

Sorry for the not very practical reply, involving experimental
unreleased code. :)

BR,
Paolo