[Haskell-cafe] Where io-streams won't fit, comparing to conduit or pipes

[Gregory Collins]

*Programming style*

Conduits and pipes use a categorical programming model in
continuation-passing style, and implement Monad instances that lift over an
arbitrary base monad. The io-streams library is not polymorphic over the
base monad, instead preferring to fix computations to IO. It's worth
pointing out that stream transformation in io-streams style is also
(implicitly) categorical in style, but in the Kleisli category for IO.

Conduits and pipes take responsibility for controlling the evaluation of
the streaming computation in its entirety, while you can either treat an
Input/OutputStream from io-streams like a Handle and feed work to it
element-wise, or chain an Input and OutputStream together using a
combinator like "connect" in streaming style.

IO-streams only does one-way streaming, while pipes and conduits can pass
data back and forth between different stream transducers.

*Resources and exceptions*

The io-streams library purposely does not do any resource management for
you, with the exception of a few "with*" functions that do some bracketing
for you. The fact that all io-streams computations run in the base IO monad
makes exception handling cheap there, but with the disadvantage that you
e.g. cannot write a stream transducer that will install an exception
handler, yield some elements, and then expect to be notified if a
downstream consumer throws an exception. In general, in io-streams, if
you're an InputStream or OutputStream that has yielded or consumed its
value, you are not guaranteed that your continuation will ever be called
again, so there is no opportunity for you to take on a cleanup action or
release some resources -- that work has to be done outside the streaming
computation. You can think of io-streams as little transducers that attach
to resources.

Since pipes and conduits use continuation-passing monads, the exception
handler problem arises there also, with the difference that since the pipes
or conduits library is running the computation, it's possible to write a
MonadCatch instance (or whatever) can carefully mask/unmask exceptions and
thread an exception handler through the monad continuation for you. The
issue with this is that every monad bind means a masking/unmasking of
exceptions and the installation/cleanup of an exception handler. Usually
people solve this by running in a ResourceT, which punts on the exception
issue for resource handling by installing one exception handler and scoping
all resource acquisition to explicit stack-like (Oleg's paper calls these
"monadic regions") contexts.

*Simplicity*

Both pipes and conduits are *much* more complicated (especially re: type
parameters), but you get additional features to go along with this
complexity so there is an engineering tradeoff there. I designed io-streams
to be as simple as possible and easy to understand for new Haskellers as
possible, while giving me of the features I needed to implement our web
programming framework.

*Correctness/Laws*

All three libraries do well on this front; pipes has had quite a bit of
formal verification done to it, and purports to follow categorical laws.
IO-streams is very simple and has 100% test coverage. Conduits is used by
quite a few people so you can expect solidity there also (but I've never
used it in anger).

*Performance*

All else being equal, an io-streams computation is likely to be slightly
faster than the others because you pay a performance penalty at bind time
for monadic polymorphism. Both pipes and conduits have leveraged their
algebraic/categorical design, however, to create RULES pragmas to do some
kinds of whole-computation stream fusion (i.e. map f . map g should get
turned into "map (f . g)"), which might lessen this disadvantage or even
eliminate it, depending on what kind of computation is happening.

Hope this helps
G

On Thu, Nov 5, 2015 at 8:33 AM, Daisuke Fujimura <me at fujimuradaisuke.com>
wrote:

> Hello cafe,
>
> Now I'm preparing a presentatio for meetup for stream processing in
> functional programming(in Tokyo), and my part is something like "why stream
> processing library".
>
> While preparing material, I encountered the question that why io-streams
> is so simple comparing others, and the reason and downsides behind its
> simplicity. I'm guessing it's due to its resource handling strategy, but
> not so confident about it because I couldn't came up with a concrete
> example yet.
>
> Do you know any good example where io-streams won't fit? Let me know if
> you have.
>
> Thanks!
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>
>


-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20151105/914c93be/attachment.html>