[Haskell-cafe] Re: Patch for library (new feature) - how to deal with it

[Maciej Piechotka]

On Fri, 2010-03-26 at 13:29 +0000, Duncan Coutts wrote:
> On Fri, 2010-03-26 at 02:11 +0000, Maciej Piechotka wrote:
> > Hello,
> > 
> > I have written patch (attached) which introduce new API for zlib -
> > personally I wrote it to have easy implementation of
> > compression/decompression in iteratee (example file - no yet working
> > -attached).
> > 
> > Earlier version of patch I tried to sent to maintainer of zlib. However
> > I received no response (possibly I get into spam folder by helpful
> > spamassassin or something like that). I definitely don't want to fork
> > this library - especially that it is in haskell platform.
> 
> Ivan is right :-)
> 

I understand. However given the last update of zlib it could be that the
address is no longer valid.

> I've discussed iterator approaches with zlib before. Tibbe and I
> discussed it in the context of web servers that want precise control
> over outputting blocks.
> 
> I'm not happy however that something that is essentially pure
> (compression/decompression) should have to have be in IO just to fit the
> iterator pattern. That seems wrong. I could tolerate ST however.
> 

Well. Iteration can be made pure again iff:
1. It will be possible to copy the z_stream. Currently it is not
possible with zlib.
2. It is deal with great care in iteration. It is possible to use safely
unsafePerformIO and seq. Iteratee can be pure as it have no copy of the
context. Existsing compress/decompress functions as well are 'safe' for
the same reason.
3. It is implemented in pure haskell.

> Secondly we cannot have the zlib package depend on the iteratee package.
> Though we can expose interfaces that allow you to construct the iterator
> stuff.
> 

Hmm. Have you seen the patch?
It does not change any dependencies. It just adds 6 functions and 3 data
structures for API (+ helper functions).

It DOES NOT remove any currently existing function (or even change
them). it is just additional interface defined. It could be a separate
package if it hasn't use Stream.

I have never imagined to add dependency of iteratee.

> I think that it is possible for you to construct the impure interface
> you want on top of the existing pure interface, using threads and MVars.
> It is a general pattern, to transform a lazy [a] -> [a] function into an
> imperative "push block / pull block" interface. You connect up the lazy
> function to an input channel and an output MVar (running in a new
> thread). Then externally you push blocks into the input channel and wait
> for blocks on the output channel. If benchmarks prove that to have too
> high an overhead then we might consider doing something lower level.
> 

1. That was my first approach. It was so horribly complicated that I
gave up. Especially if you wanted any level of error handling.

2. To fully handle the iterator API the decompress would have to be:
... -> (L.ByteString, L.ByteString)
Where first is decompressed string and second a rest of input. 

3. If anyone is going to write such function - I'm fine. I just chosen
what I believe it is a simpler and more correct approach.

> Another alternative that I would be satisfied with would be a lower
> level interface using the ST monad. That should allow the existing pure
> interface to be implemented safely on top, and hopefully would allow the
> kind of impure interface you're after.
> 

As the function have side effects I have chosen IO. The patch was
intended as a low-level interface for specific applications. If
application ensure that computation is done in proper way it may convert
it into ST/pure easily.

I think that adding unsafePerformIO/unsafeIOToST misses the point. If
computation is pure it can probably use compress/decompress. If not then
IO/ST monad does not impose additional restrictions


> Duncan
> 

Regards

PS. I would like to underline that it DOES NOT add dependency on
iteratee or any other package and it DOES NOT intend to replace/change
an existing API. It is a simply new even-lower-level API for special
situation like iteratee (web servers etc.).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20100326/cd0c1377/attachment.bin