[Haskell-cafe] ANNOUNCE: enumerator, an alternative iteratee package

[Michael Snoyman]

John,

This package looks very promising. I used iteratee for the yaml package, but
I had many of the concerns that you have mentioned below. Version 0.2 of
persistent is going to have some form of an enumerator interface for getting
the results of a query, and I eventually decided that iteratee was
introducing too much complexity to be a good candidate. However, I was able
to port the package[1] over to enumerator in about half an hour; I
especially benefited from your example applications.

The only concern that I had was the possible inefficiency of representing
all chunks as a list. In the case of persistent, the enumerator will
*always* generate a one-lengthed list, and the most common operation is
selectList, which returns all results as a list. If I used your consume
function, I believe there would be a *lot* of list traversals. Instead,
selectList[2] uses ([a] -> [a]) for building up the result internally. I
haven't really thought the issue through fully, so I can recommend anything
better. Perhaps more importantly, the simplification introduced by just
dealing with lists is well received.

Keep up the good work, I look forward to seeing more about enumerator.

Michael

[1] http://github.com/snoyberg/persistent/tree/enumerator
[2]
http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base.hs#L322
<http://github.com/snoyberg/persistent/tree/enumerator>
On Thu, Aug 19, 2010 at 7:31 AM, John Millikin <jmillikin at gmail.com> wrote:

> Most of you have probably read Oleg's essays on using left-fold
> enumerators for incremental IO. In short, by encapsulating monadic
> left-folds in an "Iteratee" type, incremental pure processing is
> possible without using lazy IO. Sources to read:
>
> Oleg: Streams and Iteratees < http://okmij.org/ftp/Streams.html >
> Magnus Therning: Trying to work out iteratees <
> http://therning.org/magnus/archives/735 >
> cdsmith: Iteratees Step By Step (Part 1) <
> http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/
> >
> John Millikin (me): Understanding Iteratees <
> http://ianen.org/articles/understanding-iteratees/ >
>
> Currently, the primary package for left-fold enumerators is John
> Lato's "iteratee". It is based on Oleg's original code, extended to
> support various forms of containers, platform-specific IO, and codecs
> for the WAV and TIFF formats.
>
> While I appreciate Mr. Lato's development of the package, I find it
> far too large, and its documentation too sparse, to effectively use.
> To correct this, I've written the "enumerator" package. It is also
> derived from Oleg's IterateeM.hs , but with a simplified API and
> significantly reduced dependency list.
>
> Hackage entry: http://hackage.haskell.org/package/enumerator
> Haddock docs: http://ianen.org/haskell/enumerator/api-docs/
> Source code (literate PDF):
> http://ianen.org/haskell/enumerator/enumerator.pdf
>
> darcs get http://ianen.org/haskell/enumerator/
>
> Additionally, I've included examples of using enumerators to implement
> simplified versions of the "cat" and "wc" utilities. These should
> serve as a useful starting point for anybody who wants to use
> enumerators in their own code:
>
>
> http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs
>
> http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs
>
> There are already a few libraries using the existing "iteratee"
> package (snap, attoparsec-iteratee, hexpat-iteratee); I am very
> interested in advice from the authors of these libraries. In
> particular, are any of the removed features (ListLike,
> WrappedByteString, seeking) something your libraries depend on? Are
> there any useful combinators you'd like to see included?
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100819/1e63e838/attachment.html