[Haskell-cafe] How does GHC read UNICODE.
ketil at malde.org
Tue May 20 03:30:57 EDT 2008
Don Stewart <dons at galois.com> writes:
> You can use either bytestrings, which will ignore any encoding,
Uh, I am hesitant to voice my protest here, but I think this bears
Bytestrings are exactly that, strings of bytes.
There are basically two interfaces, one (Data.ByteString[.Lazy]),
which operates on raw bytes (and gives you Word8s), and another
(Data.ByteString[.Lazy].Char8), which treats the contents as Chars.
The latter will only deal with Unicode code points 0..255 (or
ISO_8859-1) -- and truncate higher code point values to fit this
Basically, bytestrings are the wrong tool for the job if you need more
than 8 bits per character. I think the predecessors of bytestring
(FPS?) had support for other fixed-size encodings, that is, two-byte
and four-byte characters. Perhaps writing a Data.Word16String
bytestrings-alike using UCS-2 would be an option?
If I haven't seen further, it is by standing in the footprints of giants
More information about the Haskell-Cafe