[Haskell-cafe] Auto-termination and leftovers in Conduits

Michael Snoyman michael at snoyman.com
Sun Oct 28 04:35:17 CET 2012

The important issue here is that, when using =$, $=, and =$=, leftovers
will discarded. To see this more clearly, realize that the first line of
sink is equivalent to:

  out1 <- C.injectLeftovers CT.lines C.>+> CL.head

So any leftovers from lines are lost once you move past that line. In order
to get this to work, stick the consume inside the same composition:

sink = C.injectLeftovers CT.lines C.>+> do
    out1 <- CL.head
    out2 <- CL.consume
    return (out1, T.unlines out2)


sink = CT.lines C.=$ do
    out1 <- CL.head
    out2 <- CL.consume
    return (out1, T.unlines out2)


On Sat, Oct 27, 2012 at 9:20 PM, Myles C. Maxfield <myles.maxfield at gmail.com
> wrote:

> Hey,
> Say I have a stream of Data.Text.Text objects flowing through a
> conduit, where the divisions between successive Data.Text.Text items
> occur at arbitrary boundaries (maybe the source is sourceFile $=
> decode utf8). I'd like to create a Sink that returns a tuple of (the
> first line, the rest of the input).
> My first attempt at this looks like this:
> sink = do
>   out1 <- CT.lines C.=$ CL.head
>   out2 <- CL.consume
>   return (out1, T.concat out2)
> However, the following input provides:
> runIdentity $ CL.sourceList ["abc\nde", "f\nghi"] C.$$ sink
> (Just "abc","f\nghi")
> But what I really want is
> (Just "abc", "\ndef\nghi")
> I think this is due to the auto-termination you mention in [1]. My
> guess is that when CT.lines yields the first value, (CL.head then also
> yields it,) and execution is auto-terminated before CT.lines gets a
> chance to specify any leftovers.
> How can I write this sink? (I know I can just use CL.consume and
> T.break (== '\n'), but I'm not interested in that. I'm trying to
> figure out how to get the behavior I'm looking for with conduits.)
> Thanks,
> Myles
> [1]
> http://hackage.haskell.org/packages/archive/conduit/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20121028/5bbf58ca/attachment.htm>

More information about the Haskell-Cafe mailing list