[Haskell] Re: [Haskell-cafe] ANNOUNCE: enumerator, an alternative iteratee package

[John Millikin]

On Fri, Aug 20, 2010 at 08:59, Felipe Lessa <felipe.lessa at gmail.com> wrote:
> On Fri, Aug 20, 2010 at 12:51 PM, John Millikin <jmillikin at gmail.com> wrote:
>> Currently, I'm planning on the following type signatures for D.E.Text.
>> 'enumHandle' will use Text's hGetLine, since there doesn't seem to be
>> any text-based equivalent to ByteString's 'hGet'.
>
> CC'ing text's maintainer.  Using 'hGetLine' will cause baaad surprises
> when you process a 10 GiB file with no '\n' in sight.

This thought occurred to me, but really, how often are you going to
have a 10 GiB **text** file with no newlines? Remember, this is for
text (log files, INI-style configs, plain .txt), not binary (HTML,
XML, JSON). Off the top of my head, I can't think of any case where
you'd expect to see 10 GiB in a single line.

In the worst case, you can just use "decode" to process bytes coming
from the ByteString-based enumHandle, which should give nicely chunked
text.