implementation of UTF-8 conversion for text I/O: iconv vs hand-made

[Einar Karttunen]

On 20.04 17:38, Bulat Ziganshin wrote:
> one can find example of library that uses iconv in the "System\IO\Text.hs"
> module from http://haskell.org/~simonmar/new-io.tar.gz and example of
> hand-made encoder in module "Data\CharEncoding.hs"
> and its usage - in "System\Stream\Transformer\CharEncoding.hs"
> from http://freearc.narod.ru/Streams.tar.gz

Does Data.CharEncoding work with encodings that have state associated
with them? One example is ISO-2022-JP. Maybe with using a suitable
monad transformer?

> 2) Einar once asked me about changing the encoding on the
> fly, that is needed for some HTML processing. it is also possible that
> some program will need to intersperse text I/O with
> buffer/array/byte/bits I/O. it's a sort of things that are absolutely
> impossible with iconv 

The example goes like this:
1) HTTP client reads response from server using ascii
2) When reading headers is complete, either:
   * decode body (binary data) and after decompressing convert to text
   * decode body (text in some encoding) straight from the Handle.

Is there a reason this is impossible with iconv if the character conversion
is on top of the buffering?

- Einar Karttunen