[Haskell-cafe] Re: Patch for library (new feature) - how to deal with it

[Maciej Piechotka]

On Fri, 2010-03-26 at 17:06 +0000, Duncan Coutts wrote: 
> On Fri, 2010-03-26 at 16:50 +0000, Maciej Piechotka wrote:
> > On Fri, 2010-03-26 at 13:29 +0000, Duncan Coutts wrote:
> > > On Fri, 2010-03-26 at 02:11 +0000, Maciej Piechotka wrote:
> > > > Hello,
> > > > 
> > > > I have written patch (attached) which introduce new API for zlib -
> > > > personally I wrote it to have easy implementation of
> > > > compression/decompression in iteratee (example file - no yet working
> > > > -attached).
> > > > 
> > > > Earlier version of patch I tried to sent to maintainer of zlib. However
> > > > I received no response (possibly I get into spam folder by helpful
> > > > spamassassin or something like that). I definitely don't want to fork
> > > > this library - especially that it is in haskell platform.
> > > 
> > > Ivan is right :-)
> > > 
> > 
> > I understand. However given the last update of zlib it could be that the
> > address is no longer valid.
> > 
> > > I've discussed iterator approaches with zlib before. Tibbe and I
> > > discussed it in the context of web servers that want precise control
> > > over outputting blocks.
> > > 
> > > I'm not happy however that something that is essentially pure
> > > (compression/decompression) should have to have be in IO just to fit the
> > > iterator pattern. That seems wrong. I could tolerate ST however.
> > > 
> > 
> > Well. Iteration can be made pure again iff:
> > 1. It will be possible to copy the z_stream. Currently it is not
> > possible with zlib.
> 
> It is in principle possible, however it is rather expensive. You would
> not want to copy it in practise.
> 

I know. It is a list why I put it in IO monad. (The same for point 3)

> > 2. It is deal with great care in iteration. It is possible to use safely
> > unsafePerformIO and seq. Iteratee can be pure as it have no copy of the
> > context. Existsing compress/decompress functions as well are 'safe' for
> > the same reason.
> 
> I'm not quite sure I follow what you're referring to.
> 

While the iterating API is in IO monad you can have something like.

> > > I think that it is possible for you to construct the impure interface
> > > you want on top of the existing pure interface, using threads and MVars.
> > > It is a general pattern, to transform a lazy [a] -> [a] function into an
> > > imperative "push block / pull block" interface. You connect up the lazy
> > > function to an input channel and an output MVar (running in a new
> > > thread). Then externally you push blocks into the input channel and wait
> > > for blocks on the output channel. If benchmarks prove that to have too
> > > high an overhead then we might consider doing something lower level.
> > > 
> > 
> > 1. That was my first approach. It was so horribly complicated that I
> > gave up. Especially if you wanted any level of error handling.
> 
> On the plus side you would only have to do it once and it would let you
> wrap any lazy [a] -> [a] function into the iterator stuff.
> 

If you consider the error handling and/or trailing data it is not so
nice and for sure not general.

> > 3. If anyone is going to write such function - I'm fine. I just chosen
> > what I believe it is a simpler and more correct approach.
> 
> I understand it was easier to write it that way, however it is not a
> nice general API to expose from the zlib package

I tried to be as close as possible to Data.Digest.Pure.MD5[1]. However
due to mentioned above problems with z_stream it cannot.

> and it messes with the
> layering of the library.
> 

Could you explain?

> > > Another alternative that I would be satisfied with would be a lower
> > > level interface using the ST monad. That should allow the existing pure
> > > interface to be implemented safely on top, and hopefully would allow the
> > > kind of impure interface you're after.
> > > 
> > 
> > As the function have side effects I have chosen IO. 
> 
> Right but they only have local side effects. As such IO is overkill and
> it forces users to use unsafePerformIO/unsafeIOToST which means they
> need to know about the library internals to know if that is safe.
> 

If they would be prefixed with unsafePerformIO/unsafeIOToST user still
would have to know something about internals of library but he would
have no visual clue.

> > The patch was intended as a low-level interface for specific
> > applications. 
> 
> Sure, but I'm convinced that it is possible to make a low level ST
> interface that is not ugly and that the rest of the library can be based
> upon too.
> 

The rest of library can be based onto this (if you want I can prove it
by providing patch).

Also I'm not quite sure what you mean by ugly.

> Duncan
> 

Regards

[1]
http://hackage.haskell.org/packages/archive/pureMD5/1.1.0.0/doc/html/Data-Digest-Pure-MD5.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20100326/7118f697/attachment.bin