[Haskell-cafe] Declarative binary protocols

[Antoine Latter]

Cafe,

We have some fantastic tools for binary parsing in packages like
binary and cereal (and presumably attoparsec, which I've not used).
But they don't quite scratch an itch I have when writing
implementations of binary communication protocols.

A good example of my problem is in my implementation of the memcached
binary wire protocol: http://hackage.haskell.org/package/starling

What I've tried to do is divide up the library into a declarative
protocol description and an imperative machine to sit on a handle and
link together a server response with the source request.

In the declarative core all of the types which come off of the wire
have an associated Data.Binary.Get action - but this isn't quite good
enough. Data.Binary works on ByteStrings, but I have a handle. I don't
want to use hGetContents because I have trouble working out when lazy
IO is and is not correct. I can't use hGet because I don't know how
much to get until I'm in the middle of the Get action.

How do other folks solve this issue?

What I've done is broken down and included (getResponse :: Handle ->
IO Response) in my core protocol description module, which gets a
fixed-length header from which we can figure out how much else to get
to form the complete response. But it would be nice to have something
cleaner.

One thing I've thought of is generalizing Data.Binary.Get to operate
over either ByteStrings or Handles. But then I would be doing reads
from the handle every couple of bytes. To avoid that I could extend
the monad to include declarations of how many bytes the parser will
require, which may be declared mutliple times throughout the parser.
This felt a bit weird to me, though.

Thanks,
Antoine


Declarative core module:
http://hackage.haskell.org/packages/archive/starling/0.1.1/doc/html/Network-Starling-Core.html
IO-centric module:
http://hackage.haskell.org/packages/archive/starling/0.1.1/doc/html/Network-Starling-Connection.html