[Haskell] Re: [Haskell-cafe] ANNOUNCE: enumerator, an alternative iteratee package

[John Millikin]

On Fri, Aug 20, 2010 at 09:30, Felipe Lessa <felipe.lessa at gmail.com> wrote:
> I was thinking about an attacker, not a use case.  Think of a web
> server accepting queries using iteratees internally.  This may open
> door to at least DoS attacks.

Web servers parse/generate HTTP, which is byte-based. They should be
using the bytes-based handle enumerator.

> And then, we use iteratees because we don't like the unpredictability
> of lazy IO.  Why should iteratees be unpredictable when dealing with
> Text?  Besides the memory consumption problem, there may be
> performance problems if the lines are too short.

If you don't want unpredictable performance, use bytes-based IO and
decode it with "decode utf8" or something similar.

Text-based IO merely exists to solve the most common case, which is a
small file in local encoding with relatively short (< 200 char) lines.
If you need to handle more complicated cases, such as:

* Files in fixed or self-described encodings (JSON, XML)
* Files with unknown encodings (HTML, RSS)
* Files with content in multiple encodings (EMail)
* Files containing potentially malicious input (such as public server log files)

Then you need to read them as bytes and decide yourself which decoding
is necessary.