[Haskell-cafe] I/O and utf8

[Bulat Ziganshin]

Hello John,

Tuesday, January 10, 2006, 2:08:44 AM, you wrote:

>> i want to read a file encoded in utf8 and at a later time output portions of it
>> on the console. Is there an easy way to do this in haskell? using the standard
>> i/o functions i can read the file but the output gives me \1071 ... instead of
>> the unicode characters. 

JM> Jhc does all of its IO in utf8. CharIO is a drop in replacement for the
JM> standard prelude routines which converts everything to and from UTF8

JM> http://repetae.net/john/repos/jhc/CharIO.hs
JM> http://repetae.net/john/repos/jhc/UTF8.hs

btw, i plan to add this functionality to my Binary/Streams library,
basing on your code, John. so it will work something like:

unicode_stdout <- openWithEncoding unicode stdout
vPutStrLn unicode_stdout "it's a test"

i have the question about this issue - i also want to provide
autodetection mechanism, which relies on first bytes of text files to
set proper encoding. what is the standard rules to encode utf8/utf16
encoding used for text in file in these first bytes?



-- 
Best regards,
 Bulat                            mailto:bulatz at HotPOP.com