[Haskell] Re: [Haskell-cafe] ANNOUNCE: enumerator, an alternative
jmillikin at gmail.com
Fri Aug 20 13:07:23 EDT 2010
On Fri, Aug 20, 2010 at 09:30, Felipe Lessa <felipe.lessa at gmail.com> wrote:
> I was thinking about an attacker, not a use case. Think of a web
> server accepting queries using iteratees internally. This may open
> door to at least DoS attacks.
Web servers parse/generate HTTP, which is byte-based. They should be
using the bytes-based handle enumerator.
> And then, we use iteratees because we don't like the unpredictability
> of lazy IO. Why should iteratees be unpredictable when dealing with
> Text? Besides the memory consumption problem, there may be
> performance problems if the lines are too short.
If you don't want unpredictable performance, use bytes-based IO and
decode it with "decode utf8" or something similar.
Text-based IO merely exists to solve the most common case, which is a
small file in local encoding with relatively short (< 200 char) lines.
If you need to handle more complicated cases, such as:
* Files in fixed or self-described encodings (JSON, XML)
* Files with unknown encodings (HTML, RSS)
* Files with content in multiple encodings (EMail)
* Files containing potentially malicious input (such as public server log files)
Then you need to read them as bytes and decide yourself which decoding
More information about the Haskell-Cafe