[Haskell-cafe] [Conduit] weird action of leftover.

[Michael Snoyman]

Yes, that's a fair explanation.


On Tue, Apr 9, 2013 at 7:48 AM, Magicloud Magiclouds <
magicloud.magiclouds at gmail.com> wrote:

> Thank you for the reply. I've learnt the code of "lines". So it is because
> how ByteString works, that the conduit is not a stream of bytes, but
> chunks, right?
>
>
> On Tue, Apr 9, 2013 at 12:12 PM, Michael Snoyman <michael at snoyman.com>wrote:
>
>> It's a bug in your implementation of takeLine I believe. It doesn't take
>> into account that lines can span multiple chunks. When you call takeLine
>> the first time, you get "L1\n". leftover puts a chunk with exactly those
>> contents back. When you call takeLine the second time, it gets the chunk
>> "L1\n", and your splitAt gives you back "L1\n" and "". The "" is then
>> leftover, and the next call to takeLine gets it.
>>
>> Your takeLine needs to include logic saying "there's no newline in this
>> chunk at all, let's get the next chunk and try that." You can look at the
>> source to lines[1] for an example of the concept.
>>
>> Michael
>>
>> [1]
>> http://haddocks.fpcomplete.com/fp/7.4.2/20130313-1/conduit/src/Data-Conduit-Binary.html#lines
>>
>>
>> On Mon, Apr 8, 2013 at 8:44 AM, Magicloud Magiclouds <
>> magicloud.magiclouds at gmail.com> wrote:
>>
>>> Say I have code like below. If I comment the leftover in main, I got
>>> (Just "L1\n", Just "L2\n", Just "L3\n", Just "L4\n"). But if I did not
>>> comment the leftover, then I got (Just "L1\n", Just "L1\n", Just "", Just
>>> "L2\n").
>>> Why is not it (Just "L1\n", Just "L1\n", Just "L2\n", Just "L3\n")?
>>>
>>> takeLine :: (Monad m) => Consumer ByteString m (Maybe ByteString)
>>> takeLine = do
>>>   mBS <- await
>>>   case mBS of
>>>     Nothing -> return Nothing
>>>     Just bs ->
>>>       case DBS.elemIndex _lf bs of
>>>         Nothing -> return $ Just bs
>>>         Just i -> do
>>>           let (l, ls) = DBS.splitAt (i + 1) bs
>>>           leftover ls
>>>           return $ Just l
>>>
>>> main = do
>>>   m <- runResourceT $ sourceFile "test.simple" $$ (do
>>>     a <- takeLine
>>>     leftover $ fromJust a
>>>     b <- takeLine
>>>     c <- takeLine
>>>     d <- takeLine
>>>     return (a, b, c, d))
>>>   print m
>>>
>>> --
>>> 竹密岂妨流水过
>>> 山高哪阻野云飞
>>>
>>> And for G+, please use magiclouds#gmail.com.
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org
>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>>
>>>
>>
>
>
> --
> 竹密岂妨流水过
> 山高哪阻野云飞
>
> And for G+, please use magiclouds#gmail.com.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20130409/d54fff24/attachment.htm>