[Haskell-cafe] Sending messages up-and-down the iteratee-enumerator chain [Was: iterIO-0.1]

[dm-list-haskell-cafe at scs.stanford.edu]

At Fri, 13 May 2011 02:57:38 -0700 (PDT),
oleg at okmij.org wrote:
> 
> The code described in this message does exactly that.

Hey, Oleg.  This is really cool!  In particular, your Bindable class
has the potential to unify a whole bunch of request types and both
simplify and generalize code.  Also, Sum is clearly a more elegant
solution that just requiring everything to be Typeable.  It may solve
some problems I had where I wanted to send messages in exceptions that
contained types I didn't know to be Typeable.  I need to digest the
code a bit more, but it may make sense for me to use this technique in
a future version of iterIO.  (Much of iterIO is obviously inspired by
your stuff as it is.)

However, I still have two questions.  First, the Iter type in your
message seems more like your first iteratee implementation, which is
the approach iterIO and enumerator now take.  I wonder if it's
possible to implement something like Tell your current, CPS-based
iteratee.  Part of the reason I didn't take a CPS-based approach for
Iter was that I couldn't get the upward control requests to work.
(Also I wanted pure iteratees, which reduced the gain from CPS.)

A challenge for Tell is that you need to know the size of buffered
data and not move the input stream.  So the control handler needs to
decide what happens to residual data (since Seek does flush the
input).  Conceptually, it seems like it ought to be doable to pass
residual data up and down the enumerator/iteratee stack in a CPS
style.  But when I try to represent residual input as something like:

  data Input r m s = forall a. Input ((Stream s -> Iteratee s m a) -> m r)

I just can't get the types to work out.

The second question is what happens to residual data for downstream
requests.  In the prototype code of your message, the Stream is over
Chars, which are not a Monoid.  In practice, you obviously want
iteratees to be able to look arbitrarily far ahead--for instance an
iteratee that returns a number of digits that is a multiple of 8 might
have 8 characters of residual data (if the first 7 are digits).

So what I'm stuck on is figuring out the right way to sequence the
downstream requests with respect to the input data, particularly when
you have enumeratees transcoding from one type to the other.  Any
thoughts?

Thanks,
David