[Haskell-cafe] Re: Network.HTTP+ByteStrings Interface--Or: How to shepherd handles and go with the flow at the same time?

[Pete Kazmier]

As a newbie to Haskell, I found your thorough analysis very
interesting.  Thanks for the great read!  I have a few questions
regarding some of your comments, see below:

Jules Bean <jules at jellybean.co.uk> writes:
> E,F. Progressive GET
>     pSynGET :: URL -> ((Bool,ByteString) -> IO ()) -> IO ()
>     pAsynGET :: URL -> ((Bool,ByteString) -> IO ()) -> IO (MVar ())
> 
>     (This is a particular simple case of Oleg's iteratees, I
>     think) Download the data at whatever speed is convenient. As data
>     arrives, feed it to the 'callback' provided. The ByteString is the
>     new chunk of data, the 'Bool' is just supposed to indicate whether
>     or not this is the final chunk.

>     Incidentally there are more complex options than (Bool,Bytestring)
>     -> IO ().  A simple and obvious change is to add a return
>     value. Another is a 'state monad by hand', as in (Bool,Bytestring)
>     -> s -> s, and change the final return value of the type to IO s,
>     which allows the callback to accumulate summary information and
>     still be written as pure code. 

I want to be sure that I understand the implications of the callback
function returning an IO action as originally proposed versus it being
a pure function.  It would seem to me that if it were a pure callback
the usefulness would be limited as I would not be able to take the
data read from the network and immediately write it out to a file.  Is
this correct?

And if the above is correct, is there a way to define the callback
such that one does not have to hardcode the IO monad in the return
type so you can have the best of both worlds?

>     Other options allow the 'callback' to request early termination,
>     by layering in an 'Either' type in there. 

I believe the ability to request early termination is important, and
was one of the nice features of Oleg's left-fold enumerators.  It
would be a shame if the API did not offer this capability.

>     Another more sophisticated option, I think, is the higher rank
>
>     MonadTrans t => URL ->
>     	      	   ((forall m. Monad m) => (Bool,ByteString) -> t m)
> 		   -> t IO ()
>
>     ...which, unless I've made a mistake, allows you to write in 'any
>     monad which can be expressed as a transformer', by transforming it
>     over IO, but still contains the implicit promise that the
>     'callback' does no IO. For example t = StateT reduces to the
>     earlier s -> s example, in effect, with a slightly different data
>     layout.

I don't fully understand this, but would this prevent one from calling
IO actions as it was receiving the chunks in the callback (such as
writing it to a file immediately)?