[Haskell-cafe] ANNOUNCE: enumerator 0.4.8

[oleg at okmij.org]

Wren Thornton wrote:
> This is often conflated with the iteratee throwing an error/exception,
> which is wrong because we should distinguish between bad program
> states and argument passing.

I guess this is a matter of different points of view on exceptions. I
am a fan of the model of (effectful) computation proposed by
Cartwright and Felleisen a while ago:
	http://okmij.org/ftp/Computation/monads.html#ExtensibleDS
In their model, all the computation is done by throwing resumable
exceptions -- including the pure computation such as arithmetic and
CBV/CBN applications. The similarity of ExtensibleDS.hs with Iteratee
should be quite noticeable, especially regarding the part about
throwing `errors.'

The relation with co-routines is hard to miss: in fact, Iteratees are
built upon co-routines, which are being resumed by the enumerator. The
error message is the additional piece of data that is being associated
with `yield', telling the enumerator to do something extra rather than
mere getting the next piece of data and resuming the co-routine. The
co-routine is the simplest part of the iteratee; it is the plumbing
that takes a long time to engineer.

John A. De Goes:
> 2. Error recovery is ill-defined because errors do not describe
> what portion of the input they have already consumed;

I'm confused about this complaint: if an iteratee encounters an
unusual condition or just has a special request, it sends a message
that eventually propagates to the responsible enumerator. That
enumerator knows how much data it has sent down to iteratees. The
RandomIO module 
	http://okmij.org/ftp/Haskell/Iteratee/RandomIO.hs
is a good illustration: when the enumerator receives the seek request,
it checks if the desired stream offset corresponds to the data already
in the current IO buffer. If so, no IO is performed and the iteratee
is resumed with the existing buffer data. The tests in the file check
for that. Iteratee knows nothing about the buffer or if there is a
buffer.

Wren Thornton wrote:
> In an ideal framework the producers, transformers, and consumers of
> stream data would have a type parameter indicating the up-stream
> communication they support or require (in addition to the type
> parameters for stream type, result type, and side-effect type).
Very true. Currently the design of Iteratees quite resembles that of
Control.Exception: everything can throw SomeException. Ideally one
would like to be more precise, and specify what exceptions or sorts of
exceptions could be thrown -- by Iteratees, and by ordinary Haskell
functions. The design of a good effect system is still the topic of
active research, although there are some encouraging results.


John A. De Goes:
>  3. Iteratees sometimes need to manage resources, but they're not
>  designed to do so which leads to hideous workarounds;

Gregory Collins:
> The thing which I find is missing the most from enumerator as it
> stands is not this -- it's the fact that Iteratees sometimes need to
> allocate resources which need explicit manual deallocation (i.e.
> sockets, file descriptors, mmaps, etc), but because Enumerators are
> running the show, there is no "local" way to ensure that the
> cleanup/bracket routines get run on error.

I used to think that processing several inputs at different paces was
indeed a stumbling block. It seemed that an iteratee needed to open a
separate file, which it is indeed ill-equipped to do. Fortunately,
that difficulty has been overcome, surprisingly in a natural way with
no changes to the library:
	http://okmij.org/ftp/Streams.html#2enum1iter

The pleasant surprise is that we can iterate (no pun intended)
Iteratee monad transformers, just as we did with (IORT s) monad
transformer in the Lightweight Monadic Regions. Thus we maintain the
region-like discipline of managing resources.

I'm keen to hear of the example that seem to require Iteratee's
allocating additional resources. I'd really like to see if any of such
cases can be cast it terms of regions, implemented via iterated
Iteratee transformers.

John A. De Goes:
>     1. It does not make sense in general to bind with an iteratee that
>     has already consumed input, but there's no type-level difference
>     between a "virgin" iteratee and one that has already consumed input;

I'm not sure I follow. Why should it make a difference between a
virgin iteratee and the one that consumed some input. One should think
of the Iteratee as two arguments of fold (f and z) bundled together. Why
the function being folded over should care how many times it has been
applied to input data? It is a pure function, transforming state
plus input to a new state. The useful laws of fold hold precisely
because the function f is pure and doesn't care.

A detailed example showing why you think you need this distinction
would be appreciated.

>     4. Iteratees cannot incrementally produce output, it's all or
>     nothing, which makes them terrible for many real world problems
>     that require both incremental input and incremental output.

Again, a detailed example, a use case if you will, describing the
desired behavior is appreciated. By detailed I mean the example that
describes the desired input-output behavior, preferably including
sample input or output data. For instance, the question posed
by Evgeniy Permjacov last December was precise and helpful:
(incrementally) merge two sorted streams.

I can't promise a prompt reply this month or in April, I'm afraid.