[Haskell-beginners] Help with enumerator pipeline

[David McBride]

I wanted to solve this but the only way I could get it to compile was:

run_ ((((((enumHandle undefined undefined $= splitLines) $= parseLine)
$= filterObjects undefined) $= restrictObjects undefined) $= encoder)
$$ output)

This looks terrible, and I'm sure there is a better way, but I can't
find it.  If you remove any of the parenthesis it complains.  I would
love to heard the "right" way to do this.


On Wed, Aug 31, 2011 at 3:40 PM, Michael Xavier <nemesisdesign at gmail.com> wrote:
> I've just recently started learning how to use the enumerator library. I'm
> designing a utility that parses a file where each line is a JSON object.
> I've designed it to look like:
> source enumerator (enumHandle pretty much) ->
> chunk by lines (enumeratee) ->
> parse a line into an Object (enumeratee) ->
> filter objects based on a criteria (enumeratee) ->
> limit some keys from each object (enumeratee) ->
> encode the object into a lazy bytestring (enumeratee) ->
> output the file to stdout (iteratee)
> I'm having difficulties with the types, particularly composing the
> enumeratees in the middle. Someone in #haskell said that's a good case for
>>=> from Control.Monad but that seems to not like it if an enumeratee
> changes the type between the input and output.
> Here's the relevant types:
> type Object = Map Text Value
> pipeline :: MonadIO m => (Enumerator ByteString IO ()) -> [Text] -> [Filter]
> -> Iteratee a m ()
> pipeline s rfs fs = s $$ splitLines    >=>
>                          parseLine            >=>
>                          (filterObjects fs)   >=>
>                          (restrictFields rfs) >=>
>                          encoder              $$
>                          output
> splitLines :: Monad m => Enumeratee ByteString ByteString m b
> parseLine :: Monad m => Enumeratee ByteString Object m b
> filterObjects :: Monad m => [Filter] -> Enumeratee Object Object m b
> restrictObjects :: Monad m => [Text] -> Enumeratee Object Object m b
> encoder :: Monad m => Enumeratee Object LBS.ByteString m b
> output :: MonadIO m => Iteratee LBS.ByteString m ()
> I'm using >=> because I'd prefer to compose them left-right for readability
> Here's the error I get
>
>  Couldn't match expected type `ByteString'
>                 with actual type `M.Map Text Value'
>     Expected type: Data.Enumerator.Step ByteString m b1
>                    -> Iteratee ByteString m b2
>       Actual type: Enumeratee ByteString Object m0 b0
>     In the first argument of `(>=>)', namely `parseLine'
>     In the second argument of `(>=>)', namely
>       `parseLine
>      >=> (filterObjects fs) >=> (restrictFields rfs) >=> encoder'
> Any ideas?
> --
> Michael Xavier
> http://www.michaelxavier.net
> LinkedIn
>
> _______________________________________________
> Beginners mailing list
> Beginners at haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
>