[Haskell-cafe] Re: Strings and utf-8

[Thomas Hartman]

A translation of 

http://www.ahinea.com/en/tech/perl-unicode-struggle.html

from perl to haskell would be a very useful piece of documentation, I 
think. 

That explanation really helped me get to grips with the encoding stuff, in 
a perl context.

thomas.





Duncan Coutts <duncan.coutts at worc.ox.ac.uk> 
Sent by: haskell-cafe-bounces at haskell.org
11/29/2007 07:44 AM

To
Maurí­cio <briqueabraque at yahoo.com>
cc
haskell-cafe at haskell.org
Subject
Re: [Haskell-cafe] Re: Strings and utf-8






On Wed, 2007-11-28 at 17:38 -0200, Maurí­cio wrote:
> >>(...)  When it's phrased as "truncates to 8
>  >> bits" it sounds so simple, surely all we need
>  >> to do is not truncate to 8 bits right?
>  >>
>  >> The problem is, what encoding should it pick?
>  >> UTF8, 16, 32, EBDIC? (...)
>  >>
>  >> One sensible suggestion many people have made
>  >> is that H98 file IO should use the locale
>  >> encoding and do Unicode/String <-> locale
>  >> conversion. (...)
> 
> I'm really afraid of solutions where the behavior
> of your program changes with an environment
> variable that not everybody has configured
> properly, or even know to exist.

Be afraid of all your standard Unix utils in that case. They are all
locale dependent, not just for encoding but also for sorting order and
the language of messages.

Using the locale is standard Unix behaviour (and these days the locale
usually specifies UTF8 encoding). On OSX the default should be UTF8. On
Windows it's a bit less clear, supposedly text files should use UTF16
but nobody actually does that as far as I can see.

Duncan

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe at haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



---

This e-mail may contain confidential and/or privileged information. If you 
are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail. Any 
unauthorized copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20071129/4b87ca44/attachment-0001.htm